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ABSTRACT 


Advances  in  audio  data  compression  are  largely  driven  by  the  need  to  conserve 
transmission  rate  or  bandwidth,  while  maintaining  the  ability  to  accurately  reconstruct  the 
signal  at  the  receiver.  This  thesis  examines  data  compression  methods  with  an  emphasis 
on  techniques  for  the  compression  of  audio  data.  An  overview  of  data  compression 
schemes  is  presented  to  provide  the  background  for  a  performance  comparison  between 
selected  versions  of  data  compression  systems  featuring  adaptive  differential  pulse  code 
modulation  (ADPCM)  schemes.  Two  different  types  data  compression  systems  are 
investigated;  HR  and  FIR  impulse  systems.  A  modification  to  the  basic  ADPCM  system 
using  a  modular  function  is  implemented.  The  modular  operation  results  in  a  smaller  size 
codebook  and  prevents  data  expansion  when  the  source  is  not  matched  to  the  code.  This 
modification  is  utilized  for  both  types  of  ADPCM  coders  compared.  To  complete  the 
compression  system,  Huffinan  coding  is  employed  to  encode  and  decode  the  compressed 
data  to  and  firom  binary  form. 
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1.  INTRODUCTION 


A.  LITERATURE  REVIEW 

The  efficient  digital  representation  of  data  and  speech  signals  offers  the  possibility 
to  increase  the  data  rate  transmitted  over  existing  digital  transmission  networks  by 
providing  bit  rate  reductions  as  high  as  16:1  in  comparison  with  the  use  of  logarithmic 
pulse  code  modulation  schemes.  Data  compression  is  the  application  of  methods  to 
process  information  to  obtain  a  more  compact  representation  without  suffering 
unacceptable  loss  of  fidelity  or  accuracy  (Davidson  and  Gray,  1976).  Compression 
techniques  are  based  on  either  lossy  or  lossless  properties.  Lossless  methods  generate  an 
exact  duplicate  of  the  original  signal  upon  decompression,  whereas,  lossy  methods  trade 
complete  accuracy  for  increased  compression.  Some  of  the  lossless  algorithms  developed 
for  data  compression  include  Shannon-Fano  (Lynch,  1985),  Huffinan  (Knuth,  1985),  (Lu 
and  Gough,  1993),  and  Arithmetic  (Langdon,  1984)  schemes.  Lossy  methods  include 
various  types  of  quantization  coding,  transform  coding  and  predictive  coding  (Lynch, 
1985),  (Cappellini,  1985),  (Sibul,  1987).  Audio  (speech,  music,  etc.)  compression 
research  has  traditionally  been  separate  fi'om  other  areas.  Since  sound  is  often  an  integral 
part  of  other  data  types,  this  area  of  research  is  becoming  increasingly  more  important 
with  currently  evolving  applications.  This  thesis  examines  data  compression  techniques 
with  a  specific  emphasis  on  compressing  audio  signals  (Rabiner  and  Schafer,  1978).  The 
basic  theory  of  data  compression,  including  a  discussion  of  some  of  the  most  common 
techniques,  is  explored.  A  review  of  basic  ADPCM  coders,  along  with  an  improvement 
using  modular  arithmetic  is  presented  (Sibul,  1987),  (Einarsson,  1991).  These  latter 
techniques  are  the  structures  implemented  for  a  comparison  of  the  FIR  and  UR  versions 
of  the  modified  ADPCM  algorithms. 
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B.  DATA  COMPRESSION  BACKGROUND 


Data  Compression  is  the  process  used  to  reduce  the  physical  space  or  bandwidth 
used  to  hold  or  transmit  a  particular  set  of  data.  The  electromagnetic  spectrum,  time 
intervals  and  physical  volumes  are  all  compressible  mediums.  The  following  equation 
shows  the  interrelationship  of  all  three. 

Vohme  =f[pme  ^bandwidth  ).  (1-1) 

In  effect,  reducing  the  volume  which  a  set  of  data  occupies,  results  in  a  reduction  in 
transmission  time  or  bandwidth.  C.  E.  Shannon  originated  this  concept  in  1948  which 
initiated  the  branch  called  information  theory.  Shannon's  work  showed  that  the  extent  to 
which  a  message  can  be  compressed  and  then  accurately  restored  is  limited  by  its  entropy. 
Entropy  is  a  measure  of  the  message's  information  content:  the  more  probable  the 
message,  the  lower  its  entropy.  Entropy  can  also  be  represented  as  a  measure  of  surprise; 
the  more  unexpected  are  the  contents  of  a  message,  the  higher  its  entropy  and  vice-versa, 
which  results  in  more  bits  being  required  to  encode  it.  Shannon  used  source  entropy  and 
channel  capacity  as  the  basis  for  two  basic  theorems  which  set  precise  bounds  to  the 
accurate  representation  and  errorless  transmission  of  data  (Lynch,  1985).  With  unlimited 
resources  (time,  computing  power,  etc.),  then  the  code  wordlength  for  optimal  source 
codes  is  approximately  equal  to  (but  not  less  than)  the  source  entropy.  Hence,  source 
coding,  also  known  as  entropy  coding,  is  just  another  term  for  data  compression. 

Data  compression  methods  take  one  of  two  approaches:  lossless  or  lossy.  Lossless 
data  compression  algorithms  perfectly  reconstruct  the  compressed  data  without  error. 
Source  message  redundancy  reduction  is  the  method  these  algorithms  employ  to  achieve 
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compression.  Repetitive  data  in  a  message  set  or  signal  is  eliminated  by  sending  only  the 
changes  and  number  of  repetitions.  Conversely,  entropy  reduction  is  the  principle  used  by 
lossy  data  compression  techniques.  The  entropy  reduction  process  results  in  an 
information  loss.  Using  a  threshold  to  monitor  sample  values  is  one  example  of  a  lossy 
compression  process.  In  this  case,  compression  is  achieved  by  only  transmitting  the  time 
at  which  a  sample  value  exceeds  the  preassigned  threshold.  Data  compression  systems 
often  combine  both  lossy  and  lossless  techniques  to  achieve  maximum  compression. 

C.  THESIS  OVERVIEW 

The  current  chapter  introduces  the  basic  mechanics  of  data  compression  including 
a  review  of  the  some  of  the  relevant  literature. 

Chapter  11  describes  the  specifics  of  several  lossless  data  compression  methods. 
Huflfinan  coding,  perhaps  the  best-known  method,  used  for  encoding  and  decoding  of  the 
compressed  signals  is  featured.  Related  compression  techniques,  Shannon-Fano  and 
Arithmetic  coding  are  also  discussed. 

Chapter  III  reviews  lossy  compression  schemes.  Some  of  the  different  types  of 
quantization  schemes  are  presented.  These  include  vector  quantization  and  the  featured 
predictive  coders. 

Chapter  IV  introduces  adaptive  differential  pulse  code  modulation.  An 
improvement  to  the  basic  algorithm  designed  to  decrease  the  size  of  the  codebook  used 
for  channel  coding  is  also  explained. 

In  Chapter  V,  a  comparative  analysis  of  the  FIR  and  HR  implementations  of  the 
ADPCM  is  performed.  Specific  comparison  points  include  compression  ratio,  power 
reduction  and  speed  of  operation  of  each  design. 
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The  general  conclusion  reached  from  the  comparative  analysis  of  Chapter  V  are 
presented  in  Chapter  VI.  Topics  for  further  study  are  also  discussed. 

The  Appendix  lists  the  Matlab  code  used  to  evaluate  and  simulate  the  data 
compression  systems. 


4 


n.  LOSSLESS  COMPRESSION  TECHNIQUES 


A.  THEORETICAL  BACKGROUND 

Lossless  data  compression  algorithms  preserve  all  the  information  in  the  data  so 
that  it  can  be  reconstructed  without  error.  Adhering  to  that  constraint,  their  compression 
ratios  are  significantly  less  than  their  lossy  counterparts;  averaging  2:1  to  8:1,  depending 
upon  the  redundancy  of  the  information  source  and  the  efficiency  of  the  algorithm. 
Despite  that  fact,  some  applications  such  as  storing  or  transmitting  financial  documents, 
computer  programs  or  numerical  information,  where  a  single  bad  bit  could  be 
catastrophic,  demand  lossless  techniques.  A  number  of  different  lossless  compression 
methods  with  many  variations  exist.  These  techniques  fall  into  the  following  general 
categories:  optimum  source  coding,  nonredundant  sample  coding  and  binary  source 
coding.  Again,  note  that  coding  techniques  result  in  data  compression,  thus  their 
discussion  is  germane  to  a  data  compression  exploration.  Nonredundant  sample  coding 
(NSC)  makes  use  of  threshold  values  and  sends  only  time  information.  The  dominant 
type  of  NSC  is  run-length  coding  and  others  include  predictors  and  interpolators  (Lynch, 
1985).  Since  the  timing  information  sent  by  this  method  is  asynchronous,  buffering  is 
required.  The  compression  systems  studied  in  this  thesis  are  synchronous  ADPCM 
systems  without  buffering,  and  no  further  discussion  of  NSC  occurs.  The  reminder  of 
this  chapter  discusses  optimum  source  and  binary  coding  methods. 

Optimum  source  encoding  starts  with  statistically  independent  samples  and  codes 
them  in  such  a  way  as  to  make  the  average  word  length  equal  to  the  sample  entropy.  This 
method  is  also  called  entropy  coding  (since  the  code  approaches  the  entropy  of  the 
source).  Entropy  relates  to  randomness.  Ifthe  contents  ofa  message  are  unexpected. 
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then  the  entropy  is  high.  If  the  contents  of  a  message  are  as  expected,  then  entropy  is 
low.  If  the  entropy  of  a  data  set  is  reduced,  the  lower  entropy  data  set  can  then  be 
encoded  with  fewer  bits  resulting  in  data  compression.  Two  main  methods  occupy  this 
category:  Shannon-Fano  and  HufiBnan  coding. 

B.  SHANNON-FANO  CODING 

The  Shannon-Fano  coding  procedure  produces  binary  codes  that  are 
instantaneously  decodable.  An  explanation  of  instantaneous  decodability  is  provided 
later.  One  application  of  this  technique  is  as  a  stage  of  the  well-known  PKZIP's  (Apiki, 
1991)  "imploding"  algorithm.  Shannon-Fano  code  reaches  an  efficiency  of  100%  only 
when  the  source  message  probabilities  are  negative  powers  of  two.  The  following  coding 
procedure  and  example  detail  and  illustrate  the  process  (Lynch,  1985):  1)  Arrange  the 
source  message  probabilities  in  descending  order,  2)  Divide  the  message  set  into  two 
subsets  of  equal,  or  almost  equal,  total  probability  and  assign  a  zero  as  the  first  code  digit 
in  one  subset,  and  a  one  as  the  first  code  digit  in  the  second  subset,  3)  Continue  this 
process  until  each  subset  contdns  only  one  message.  The  accompanying  example  shows 
how  the  efficiency  is  computed. 

EXAMPLE 

The  following  set  of  messages  with  probabilities,  Pj,  is  given: 
m(i)  1  2  3  4  5  6  7 

Pi  0.4  0.1  0.1  0.1  0.1  0.1  0.1 

The  average  codeword  length,  Lavg,  is  computed  by  the  formula: 


6 


(2.1) 


Lavg-T,l(m(i))P. 


>1 


where  Mis  the  number  of  messages  in  the  set  and  represents  the  number  of  bits  in 

the  code  for  symbol  m{i).  Applying  the  procedure  outlined  earlier  the  following  Shannon- 
Fano  code  would  be  generated  as  follows: 

Split  the  seven  messages  into  two  groups  with  their  probabilities  being  as  equal  as 
possible.  Thus,  group  one  would  be  message  one  and  message  two  with  a  total  P,  =  0.5, 
and  group  two  would  be  the  four  remaining  messages,  mS-m7,  with  the  same  total  P,-.  The 
next  step  would  be  to  arbitrarily  assign  a  ‘  1'  to  group  one  and  ‘O'  to  group  two.  The 
process  would  be  repeated;  group  one  would  be  split  into  two  groups  and  a  second  ‘T 
would  be  arbitrarily  assigned  to  message  one,  ml,  and  a  ‘O'  to  m2.  This  process  is 
arbitrary  because  it  could  have  been  done  in  the  reverse,  assigning  a  ‘O'  to  ml  and  a ‘T  to 
m2.  Likewise,  group  two  would  be  split  into  two  subgroups,  m3  and  m4  in  one  group  and 
m5-m7  in  another  group.  Again,  each  member  of  the  first  subgroup  would  be  assigned  a 
one  and  each  member  of  the  second  subgroup  would  append  a  zero  onto  its  current  code. 
As  per  the  procedure,  the  process  would  continue  until  each  message  had  been  assigned  a 
unique,  instantly  decodeable  codeword.  The  entire  process  is  shown  in  Table  2.1.  Using 
Equation  2. 1,  Lavg  is  computed  to  be  2.7  bits: 

[2(0.4)  +  2(0.1)  +  3(0.1)  +  3(0.1)  +  3(0.1)  +  4(0.1)  +  4(0.1)  =  2.7  bits]. 

By  comparison,  the  entropy  is  given  by: 
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(2.2) 


E-ZPIog^, 

U1 


Original 

Group 

New 

Grp 

New 

Grp 

Org. 

msg 

m(i) 

Code 

Code 

Code 

mil) 

Pi 

Code 

ml2 

1 

mJ 

11 

ml 

11 

ml 

0.4 

11 

m2 

10 

m2 

10 

m2 

0.1 

10 

m34567 

0 

m34 

01 

m3 

on 

m3 

0.1 

on 

m4 

010 

m4 

0.1 

010 

m567 

00 

m5 

001 

mS 

0.1 

001 

m67 

000 

m6 

0.1 

0000 

m7 

0.1 

0001 

TAB 


/E  2.1  Shannon-Fano 


Codeword  Construction 


This  value  is  computed  to  be  2.522  bits  resulting  in  an  efficiency  of  93.4%,  where 
efficiency,  r\,  is  given  by: 


^  = 


H 

Lavg 


(2.3) 


Shannon-Fano  coding  has  a  close  relative,  called  Huffinan  coding,  which  is  well  known  to 
have  a  significantly  greater  efficiency.  Increased  efficiency  is  one  reason  Huffinan  is  a 
preferred  technique. 
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c. 


HUFFMAN  CODING 


1.  Static  Huffman  Coding 

Huflfinan  coding  is  probably  the  best-known  method  of  data  compression  and  has 
many  practical  applications.  These  include  the  last  stage  of  JPEG  compression  and  the 
MNP-5  modem  data  compression  standard  (Apiki,  1991).  The  basic  premise  of  Huflftnan 
coding  is  the  creation  of  a  binary  tree  with  internal  nodes,  called  branches,  that  represent 
the  path  to  external  nodes,  known  as  leaves.  There  is  one  leaf  on  the  tree  for  each  sjnnbol 
in  the  data  set.  These  leaves  are  combined  by  connecting  them  to  branches.  The  tree 
begins  at  a  branch  called  the  root,  which  has  no  number  assigned  to  it  and  has  a  probability 
of  one.  The  binary  numbers  encountered  along  the  path  to  the  leaves  comprise  the 
variable-length  codes  for  each  symbol  of  a  given  sequence,  with  each  code  being 
represented  by  an  integral  number  of  bits.  Symbols  with  higher  probabilities  are  given 
shorter  bit  codes  while  symbols  with  lower  probabilities  are  assigned  longer  bit  codes  and 
thus,  longer  branches  on  the  tree.  The  Huffinan  tree  is  constructed,  after  determining  the 
frequency  of  occurrence  or  probability  for  each  symbol  in  a  source,  by  repeatedly 
combining  the  two  least  probable  s5mibols  at  each  stage.  This  process  continues  until  the 
original  source  is  reduced  to  only  two  sjmibols.  These  two  symbols  are  respectively 
assigned  the  bit  values  of  'O'  and  '1'.  The  codes  for  the  previous  reduced  stage  are  then 
determined  by  appending  a  'O'  or  a 'T  to  the  right  of  the  code  corresponding  to  the  two 
least  probable  symbols,  and  so  on.  Once  each  symbol  in  the  original  source  is  assigned  a 
binary  code,  the  Huffinan  coding  is  complete.  Table  2.2  shows  an  example  source 
reduction  and  Table  2.3  performs  the  resulting  codeword  construction  for  generating  the 
Huffinan  code  (Apiki,  1991),  (Langdon,  1984). 
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Table  2.2  displays  a  source  reduction  process  for  a  source  with  five  symbols;  ml  through 
m5.  Symbol  probability  is  determined  by  dividing  the  total  number  of  occurrences.  No. 
occ,  of  each  S5m.bol  by  the  total  number  of  symbols.  The  two  lowest  probability  symbols 
are  combined  to  make  a  new  symbol  with  a  probability  equal  to  that  of  the  combined 
symbols.  The  next  two  lowest  probabilities  are  then  combined.  When  symbols  of  equal 
probability  are  encountered,  they  can  be  combined  in  any  order;  with  an  optimal  code  still 
resulting.  The  Huffinan  code  is  optimal  in  that  it  produces  a  code  with  the  minimal 
average  word  length.  Thus,  in  Table  2.2,  m5  could  have  been  combined  with  m3,  or  m2, 
not  just  m4.  No  ambiguity  results  from  the  arbitrary  combinations  since  each  symbol  is 
given  a  unique  binary  code.  The  construction  of  this  code  is  depicted  in  Table  2.3.  The 
combination  process  continues  until  only  two  symbols  remain.  These  symbols  are 
assigned  the  codes  of  *0'  and  shown  in  Table  2.3.  The  reconstruction  occurs  from  the 
right  side  of  the  table,  starting  with  the  final  two  reduced  source  symbols  from  Table  2.2. 


Origir 

lal  source 

New 

symbol 

Symbol 

Prob. 

New 

symbol 

Symbol 

Prob. 

New 

symbol 

Symbol 

Prob. 

m(J) 

No 

occ 

Prob 

ml 

20 

0.4 

ml 

0.4 

ml 

0.4 

m2345 

0.6 

m2 

10 

0.2 

m2 

0.2 

m345 

0.4 

ml 

0.4 

m3 

10 

0.2 

m3 

0.2 

m2 

0.2 

m4 

6 

0.2 

m45 

0.2 

mS 

D 

0.08 

TABLE  2.2  Huffinan  Source  Reduction  Procedure 
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The  higher  probability  symbol  is  arbitrarily  assigned  the  value  of  'O'  in  Table  2.3. 
It  could  have  been  assigned  a  T  and  the  lower  probability  symbol  could  have  received  a 
'O'.  The  symbol  which  is  a  combination  of  symbols  (m2345)  is  broken  into  two  symbols 
meaning  there  are  now  three.  A  'O'  or  a 'T  is  appended  to  the  right  of  the  original  code  of 
the  two  least  probable  symbols,  following  whichever  convention  was  established  with  the 
first  two  symbols,  ml  and  m2345.  This  process  continues  until  a  codeword  is  generated 
for  each  of  the  five  original  S)nnbols.  Table  2.3,  which  shows  how  to  construct  a 
codeword,  must  also  be  transmitted  to  the  receiver  so  that  the  receiver  may  correctly 
decode  received  messages. 

From  Table  2.3,  it  is  easily  seen  that  the  final  codewords  have  the  unique  prefix 
property,  thus  no  single  code  is  a  prefix  (or  subset)  for  another  code.  This  is  what  is 
meant  by  instantaneously  decodable.  Each  symbol  can  be  transmitted  and  immediately 
decoded  without  confusion  with  any  other  symbol  as  the  symbols  arrive  because  the 
decompressor  must  already  have  a  copy  of  the  probability  table.  In  communication 
channel  applications,  Huffman  compression  is  further  limited  since  a  copy  of  the 
probability  table  must  be  transmitted  with  the  compressed  data  to  allow  decompression  at 
the  receiver  (Cappelini,  1985).  As  the  decompressor  receives  code,  it  processes  it  in 
reverse.  The  decompressor  starts  at  the  tree’s  root  and  follows  the  sequence  of  incoming 
‘I's  and  ‘O's  through  the  tree  until  it  reaches  a  leaf  The  symbol  attached  to  that  leaf  is  the 
decoded  character  and  the  next  bit  received  obviously  starts  a  new  character.  Therefore, 
there  is  no  need  for  the  receiver  to  explicitly  know  the  length  of  each  symbol  (Apiki, 
1991).  The  requirement  that  the  number  of  bits  for  each  code  must  be  an  integer  is  a 
restriction  on  the  Huffman  codes  efficiency.  The  ideal  binary  code  length  for  a 
symbolw(/),  given  by; 
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l(m(i))=-log^(i). 


(2.4) 


where  I  is  the  wordlength  and  P(  i)  is  the  probability  of  symbol  7w(  i),  is  achieved  only 
when  its  probability  is  a  negative  power  of  two,  such  as  1/2,  1/4,  1/8,  and  so  on,  which 
results  in  integer  values  for  the  wordlengths.  In  that  case,  no  bits  would  be  wasted  while 
representing  decimal  values  like  2.67  bits,  for  example,  since  that  word  length  would 
require  three  bits  wasting  part  of  the  third  bit. 


Original  source 

Reduced 

source 

Reduced 

source 

Reduced 

source 

m(i) 

Codeword 

ml 

1 

ml 

1 

ml 

1 

m2345 

0 

m2 

01 

m2 

01 

m345 

00 

ml 

1 

m3 

000 

m3 

000 

m2 

01 

m4 

0010 

m45 

001 

mS 

0011 

TABLE  2.3  Huffinan  Codeword  Construction  Procedure 


Thus,  the  chance  of  the  Huffinan  code  being  set  to  ideal  lengths  is  unlikely.  The 
example  described  in  Tables  2.2  and  2.3  accomplishes  bit  compression  by  reducing  the 
average  symbol  length  (Zovg),  Equation  2.5,  fi-om  three  to  2.52. 

Lavg=llJ(m(i))P^  (2.5 

The  original  average  symbol  length  is  three  because  three  binary  bits  are  required  to 
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differentiate  between  five  symbols.  The  calculation  of  the  compressed  Lavg,  using 
probability  values  from  Table  2.2  and  message  lengths  from  Table  2.3  is: 

[Lavg  =  1(0.4)  +  2(0.2)  +  3(0.2)  +  4(0.2)  +  4(0.08)  =  2.52  bits]. 

2.  Dynamic  Huffman  Coding 

A  dynamic  version  of  Hufiman  compression  can  construct  the  Huffman  encoding 
tree  on  the  fly  while  reading  and  actively  compressing,  effectively  eliminating  the  above 
efficiency  limitations.  The  encoding  tree  is  continually  corrected  to  reflect  the  changing 
probabilities  of  the  input  data  (Knuth,  1985).  Therefore,  instead  of  first  determining 
probabilities  and  then  encoding  as  in  static  Huffman,  the  adaptive  model  initially  assumes 
all  symbol  weights  are  zero  and  counts  the  symbol  frequencies  as  it  encodes  them.  After 
reading  each  symbol,  the  Huffman  code  is  modified  to  account  for  the  new  character. 
Similarly,  the  decoder  learns  the  symbol  frequencies  and  updates  the  Huffinan  code  in  the 
same  manner.  The  encoder  and  decoder  remain  s3mchronized  because  any  changes  to 
symbol  probabilities  in  the  encoder  also  occur  in  the  decoder.  The  number  of  different 
symbols  must  be  sent  to  the  receiver  to  allow  decoding  of  the  compressed  data.  The 
adaptive  Huffinan  tree  is  generated  starting  with  an  uninitialized  tree  and  an  empty  leaf, 
which  represents  a  node  with  no  symbol  attached  to  it,  of  zero  probability.  The  following 
example,  (Apiki,  1991),  demonstrates  the  process.  The  input  symbols  are:  “This  is”, 
including  the  space. 

The  example  shows  that  the  tree  starts  with  the  empty  leaf,  eo  and  then  sends  the 
actual  ASCII  character  the  first  time  it  encounters  a  symbol,  generating  a  code  for  each 
symbol  in  the  process.  Therefore,  when  the  T  is  received  the  tree  looks  like  the  left-most 
structure  in  Figure  2.1.  The  circle  represents  the  root  and  if  only  a ‘T’  were  being 
transmitted,  then  the  ASCII  character  for  a ‘T’  would  be  actually  sent.  By  the  time  the 
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entire  word  This  has  been  seen  by  the  encoder  the  tree  has  been  changed  to  match  the 
structure  second  from  the  left  in  Figure  2.1.  The  second  time  a  symbol  appears,  its  code  is 
transmitted  instead  of  its  literal  symbol.  As  the  frequency  of  a  symbol  increases,  in  this 
case  the  symbols  ‘i’  and  ‘s’,  it  move  higher  up  the  tree  towards  the  root. 


Innut!  Thisrs■n^^ee^ic 


eo  (sp8ce)i  eo  (space)i 

Tree  after  T  Afters  After  (space)  Final  Tree 

Output:  TOhOOil 00s000(space)01  111 
Figure  2.1  Dynamic  Huffinan  Tree  Construction  (Apiki,  1991) 


The  initial  tree,  held  by  both  the  compressor  and  decompressor,  has  only  the 
root  and  a  single  empty  leaf,  Cg.  The  compressor  starts  the  process  by 
reading  in  a  character.  It  attaches  this  character  to  the  1-branch  of  the  root, 
leaving  the  empty  leaf  on  branch  ‘O'.  It  then  sends  this  character  to  the 
decompressor  as  a  literal  ASCII  code,  and  the  decompressor  make  the  same 
adjustment  to  its  tree. 

For  each  character  read  thereafter,  the  compressor  performs  the 
following  steps.  First,  it  check  to  see  if  the  code  is  in  the  encoding  tree.  If  the 
code  is  there,  the  compressor  sends  it  in  the  same  fashion  as  in  the  static  case. 
If  not,  it  sends  the  code  for  the  empty  leaf.  Then  it  sends  the  new  character 
as  a  literal  ASCII  code.  Finally,  the  compressor  adds  two  codes,  one  for  a 
new  empty  leaf  on  branch  0  and  one  for  the  new  code  on  branch  1 .  When  the 
tree  is  full,  the  compressor  just  changes  the  last  empty  leaf  node  into  the  last 
character  (Apiki,  1991). 
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In  the  example  above,  there  are  four  trees  shown.  Each  tree  is  labeled  underneath, 
showing  what  the  last  character  processed  was.  The  intermediate  trees  which  show  the 
current  tree  structures  after  the  ‘h’,  and  the  first  and  second  ‘i’s  are  not  shown.  The 
compressor  starts  out  with  just  the  empty  leaf  and  then  reads  in  character ‘T’. 

The  ‘T’  is  placed  on  branch  ‘  T  and  the  empty  leaf  is  placed  on  branch  ‘O'.  As  the  next 
character,  ‘h’  is  read  in,  the  procedure  in  the  quote  above  is  followed.  Since  the  ‘h’  is  not 
already  in  the  table,  the  code  is  sent  for  the  empty  leaf  and  the  ASCII  code  for  an  ‘h’  is 
transmitted.  Then  the  compressor  builds  a  new  tree,  with  the  empty  leaf  attached  to  the 
root  by  branch  ‘O'  and  the  other  two  symbols  ‘T’  and  ‘h’  attached  to  the  root  by  branch  ‘T 
which  will  have  a  weight  of  two;  one  for  the  number  of  occurrences  of  each  symbol.  The 
process  continues  with  each  new  symbol.  The  numerals  in  Figure  2. 1  indicate  the  weights 
of  the  nodes  below  it.  This  value  must  be  updated  so  that  when  a  node  has  a  weight 
higher  than  a  node  above  it,  the  two  nodes  are  swapped,  vdth  the  child  nodes  remaining  in 
their  same  places.  This  is  observed  in  the  final  structure  where  the  ‘s’  with  a  weight  of 
two  has  been  swapped  to  a  higher  node  with  the  ‘T’  of  weight  one.  These  swaps  are 
necessary  to  make  the  tree  adapt  to  the  changing  probabilities  of  the  data  symbols.  Again, 
note  that  each  character  is  transmitted  as  it  is  read  in  and  then  a  new  tree  structure  defined 
which  allows  the  decompressor  to  simultaneously  make  the  same  changes  to  its  tree.  The 
final  output  is  shown  on  Figure  2. 1  and  the  codes  are  deciphered  as  follows;  The  literal 
ASCII  code  for  a  ‘T’  is  transmitted  for  the  ‘T’,  then  the  code  for  the  empty  leaf,  ‘O'  and 
the  literal  code  for  ‘h’  are  sent  for  the  ‘h’,  next  the  new  code  for  the  empty  leaf ‘00'  is  sent 
along  with  the  literal  code  for  the  first  ‘i’,  100s,  where  100  is  the  new  code  for  the  empty 
leaf  is  sent  for  the  first ‘s;  OOO(space)  represents  the  space,  then  the  Huffman  code  ‘OT  is 
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transmitted  for  the  second  ‘i’  and  the  final  ‘  1 11'  is  the  code  for  the  second  ‘s’. 

The  other  major  type  of  lossless  data  compression  explored  is  binary  source 
coding.  Though  binary  source  codes  are  typically  grouped  as  redundancy-reduction 
methods,  and  thus  usually  require  buffering,  the  next  technique  under  consideration  is  very 
similar  to  Huffman  coding.  Its  advantages  over  Huffman  are  presented  next. 

D.  ARITHMETIC  CODING 

Though  Huffman  coding  is  largely  considered  the  most  efficient  fixed-length 
lossless  coding  method,  it  has  one  major  disadvantage.  That  is  the  requirement  that 
symbol  codes  be  an  integral  number  of  bits.  As  earlier  stated,  this  only  occurs  for 
probabilities  which  are  a  negative  power  of  two.  If  the  symbol  probability,  is  1/5,  for 

example,  the  optimum  code  length  is  given  by: 

-log2(PJ‘2.32  bits.  (2.6) 

Huffman  code  would  use  two  or  three  bits  to  encode  the  symbol.  Clearly  the  compression 
is  neither  completely  efficient  nor  maximum.  Arithmetic  coding  provides  a  viable  solution 
to  this  limitation.  This  technique  represents  the  entire  message  as  a  number  stream 
(Langdon,  1984).  The  entire  symbol  domain  is  encompassed  on  the  interval  of  real 
numbers  between  zero  inclusive  and  one  exclusive,  [0,  1).  Each  symbol  is  assigned  a 
range  within  the  interval,  corresponding  to  its  probability. 
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EXAMPLE  1 


Table  2.4  demonstrates  a  sample  interval  range  assignment.  The  initial  range  is  [0,  1). 

The  initial  range  is  divided  up  amongst  the  symbols  starting  with  the  highest  probability 
symbol  T.  The  T  is  assigned  the  interval  [0.0,  0.4),  next  the  second  symbol,  'T',  is 
assigned  twenty  five  percent  of  the  total  range  [0,  1),  starting  where  the  last  symbol's 
interval  ended,  0.4,  since  the  symbol  intervals  cannot  overlap.  The  procedure  is  followed 
until  all  symbols  have  been  assigned  a  portion  of  the  initial  range,  [0,  1).  Table  2.5 
demonstrates  the  encoding  of  the  word  'timing'  based  on  the  range  assignments  of  Table 
2.4.  The  first  symbol  to  be  coded  assumes  the  same  range  on  the  initial  interval,  [0,  1),  as 
it  is  assigned  in  Table  2.4.  Therefore,  the  current  range  is  now  [0.4,  0.65). 

As  each  additional  symbol  is  processed,  the  range  is  narrowed  to  that  interval 
Avithin  the  current  range  which  is  allocated  to  the  symbol.  So  when  the  T  is  to  be 
processed  its  probability,  P„  is  multiplied  by  the  current  range  producing  the  new  range  of 
0.1.  [0.4*0.25  =  0.1].  The  greater  a  symbol's  probability,  the  less  it  will  reduce  the 
current  range;  resulting  in  fewer  bits  being  added  to  the  code.  This  can  be  seen  firom  the 
bracketed  Equation  above  where  the  P,  for  an  T’  is  0.25,  which  means  the  previous  range 
is  reduced  to  25%  of  the  initial  range  of  0.4  or  0.1.  The  choice  of  where  to  place  the  new 
interval  within  the  range  of  [0.4,  0.65)  is  arbitrary.  Thus,  the  new  interval  could  be  [0.49, 
0.59)  or  any  other  0.01  interval  within  [0.4,  0.65).  In  Table  2.5,  the  new  range  was 
chosen  to  be  [0.4,  0.5).  When  the  "M  is  processed,  its  probability  is  multiplied  by  the 
current  range  resulting  in  a  new  range  of  0.01  [0.1  *  0.1  =  0.01].  Again,  this  0.01  interval 
is  arbitrarily  chosen  from  the  entire  range  of  [0.4,  0.5).  No  confusion  results  from  the 
arbitrary  range  selection  because,  just  like  in  Huffman  coding,  the  coding  table  must  be 
sent  to  the  receiver  to  decode  messages.  The  process  continues  until  one  decimal  number 
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is  determined  which  represents  the  entire  message,  'timing'.  The  decoding  process  is  then 
fairly  straightforward. 

The  first  symbol  is  determined  fi-om  the  sub-interval,  of  the  initial  range,  [0,  1)  in 
which  the  encoded  message  falls.  Since  the  received  value  is  on  the  interval  of  [0.483 14, 
0.48320),  it  can  be  seen  that  the  first  symbol  came  fi'om  the  sub-interval  [0.4,  0.65)  and 
must  be  a 'T'.  The  next  symbol  is  determined  by  subtracting  from  the  received  encoded 
value,  0.483 14  in  Table  2.6,  the  low  value  from  Table  2.4,  of  the  first  symbol  and 
dividing  by  the  probability,  P,,  of  the  first  symbol's  range,  0.25.  The  symbol  is  then  found 
via  the  interval  in  which  the  new  encoded  value  falls.  In  Table  2.5,  the  following  values 
result:  [  (0.483 14  -  0.4)70.25  =  0.33256  ].  This  new  value  falls  within  the  range  [0.0,  0.4) 
indicating  the  second  symbol  is  an  T.  Results  for  the  remaining  decoding  of  symbols  are 
illustrated  in  Table  2.6.  Note  that  in  actual  coding,  the  values  of  the  encoded  numbers  will 
be  represented  in  binary.  Decimal  values  were  utilized  in  the  above  example  to  assist  in 
concept  understanding.  Since  the  decoder  interprets  the  encoded  number  0.0  as  a  symbol 
(T  in  Table  2.  4)  in  the  domain  interval,  an  end  of  message  symbol  must  be  transmitted 
with  the  code. 


Symbol 

Probability 

Range 

I 

0.40 

[0.00,  0.40) 

T 

0.25 

[0.40,  0.65) 

N 

0.15 

[0.65,  0.8) 

M 

0.10 

[0.80,  0.9) 

G 

0.10 

[0.90,  1.0) 

TABLE  2.4  Arithmetic  Coding  Range  Assignment  for  Example  1 
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Symbol  Number 

Symbol 

Low  Value 

High  Value 

2 

T 

0.40 

0.65 

1 

I 

0.40 

0.50 

4 

M 

0.48 

0.49 

5 

I 

0.480 

0.484 

1 

N 

0.4826 

0.4832 

3 

G 

0.48314 

0.48320 

TABLE  2.5  Arithmetic  Encoding  Process  for  Example  1 


EXAMPLE  2 

Assume  a  stream  of  VYWYW’s '  is  to  be  compressed.  The  probability  of  V  is  known  to 
be  0.9,  while  the  end-of-message  character  has  a  probability  of  0.1.  The  range  [0,  0.9)  is 
assigned  to  the  letter  V  and  [0.9, 1.0)  is  assigned  to  the  end-of-message  character.  Table 
2.7  displays  the  results  (Langdon,  1984). 


Encoded  Number 

Symbol 

Low 

High 

Range 

0.48314 

T 

0.40 

0.65 

0.25 

0.33256 

I 

0.00 

0.40 

0.40 

0.8314 

M 

0.80 

0.90 

0.10 

0.314 

I 

0.00 

0.40 

0.40 

0.785 

N 

0.65 

0.80 

0.15 

0.90 

G 

0.90 

1.00 

0.10 

TABLE  2.6  Arithmetic  Decoding  Process  for  Example  1 
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New  Character 

Low  Value 

High  Value 

V 

0.0 

0.9 

V 

0.0 

0.81 

V 

0.0 

0.729 

V 

0.0 

0.6561 

V 

0.0 

0.59049 

V 

0.0 

0.531441 

V 

0.0 

0.4782969 

END  OF  FILE 

0.43046721 

0.4782979 

TABLE  2.7  Arithmetic  Encoding  for  Example  2 


The  value  0.4782979  would  then  be  transmitted  to  represent  a  string  of  nine  Vs 
rather  than  sending  nine,  eight  bit,  ASCII  characters.  Like  Huffinan  codes,  arithmetic 
codes  have  a  dynamic  version  and  often  use  a  zero-order  Markov  model;  though  higher- 
order  models  can  be  implemented.  The  major  problem  with  arithmetic  coding  is  that  most 
computers  cannot  process  numbers  of  the  length  needed  to  encode  certain  data  types,  i.e. 
images.  This  is  can  be  seen  in  the  example  of  Table  VII  in  that  the  precision  required  to 
represent  only  a  few  more  ‘V’s  would  result  in  a  precision  greater  than  the  normal 
computer  could  achieve.  This  problem  is  overcome  by  only  sending  a  portion  of  a 
message  or  data  set  which  can  be  represented  within  the  precision  of  the  computer  system 
sending  it  (Weiss  and  Schremp,  1993). 

Another  drawback  is  that  of  loss  of  precision  between  the  high  and  low  values  as 
the  ranges  gets  very  small.  This  often  results  in  the  low  value  being  higher  than  the  high 
value  and  consequently,  causing  overflow.  Inserting  checks  into  the  process  prevent  this 
problem  at  the  expense  of  greater  complexity  (Langdon,  1984). 


20 


m.  LOSSY  DATA  COMPRESSION  TECHNIQUES 


A.  LOSSY  COMPRESSION  THEORETICAL  BACKGROUND 

Lossy  compression  describes  processes  where  information  is  irretrievably  lost.  It 
is  typically  used  for  applications  where  there  is  a  notion  of  fidelity  associated  with  the 
data.  Such  applications  often  involve  digitally  sampled  analog  data  (still  images,  video, 
etc.)  where  it  is  only  necessary  that  the  decompressed  data  be  acceptably  close  in  quality 
to  the  original.  Thus,  it  is  very  useful  for  audio  compression  applications  where  the 
human  ear  is  not  discerning  enough  to  detect  the  loss.  Lossy  compression  techniques  are 
a  subset  of  the  entropy  reduction  class  of  data  compression.  The  major  types  of  lossy 
compression  methods  use  some  form  of  quantization.  Efficient  quantizer  design  requires 
an  accurate  statistical  model  of  the  data  source.  For  this  thesis,  linear  prediction  is  used  to 
model  the  data. 

There  are  three  general  categories  of  quantization  compression:  1)  Zero-memory, 
2)  Block  and  3)  Sequential.  The  distinction  between  these  categories  is  not  strong,  as 
techniques  which  fall  under  one  category  also  fit  into  another.  Zero-memory  quantization 
is  the  process  where  samples  are  quantized  individually,  while  in  block  quantization,  a 
block  of  input  samples  are  represented  by  a  block  of  output  values  chosen  fi'om  a  finite  set 
of  possible  output  blocks.  Finally,  sequential  quantization  quantizes  input  samples  using 
information  from  its  surrounding  samples  on  a  block  or  non-block  basis  (Knuth,  1985). 
Thus,  the  sequential  techniques  can  all  also  be  block  quantization  techniques.  Under  the 
zero-memory  category  are  vector  and  scalar  quantization.  Multi-path  search  coders  and 
predictive  coders  are  the  dominant  form  of  sequential  quantization  techniques,  which  as 
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stated  earlier  can  also  be  used  in  a  block  quantization  form.  The  block  quantization 
category  encompasses  various  types  of  transform  coding.  Transform  coding  operates  on  a 
block  of  n  samples  generating  an  output  sequence  by  matching  the  input  block  with  its 
closest  approximation  from  a  codebook  of  sequences.  The  compression  scheme  is 
optimized  by  minimizing  the  difference  between  the  two  sequences.  See  Lynch  (1985), 
and  Jayant  and  Noll  (1984)  for  additional  information  on  transform  coding.  No  further 
discussion  of  transform  coding  is  presented  here. 

B.  SCALAR  AND  VECTOR  QUANTEAHON 

For  the  zero-memory  method,  each  data  sample  is  quantized  independently  of  all 
other  samples  using  the  same  quantizer,  therefore,  the  system  has  no  memory  of  previous 
data.  The  compression  algorithm  is  optimized  to  minimize  the  quantizing  noise  using 
fixed  quantizing  levels.  Another  name  for  zero  memory  quantization  is  scalar 
quantization.  Scalar  quantization  is  just  a  special  case  of  a  very  powerful  data 
compression  technique  called  vector  quantization.  Vector  quantization  is  a  process  based 
on  a  codebook.  A  codebook  is  a  collection  of  vectors  or  lists  of  “typical”  data  sequences. 
The  codebook  vectors  are  very  similar  to  the  branches  in  a  tree  structure  used  for  tree  or 
trellis  coding  which  will  be  discussed  later.  The  codebook  vectors  contain  the  parameters 
used  to  reproduce  the  original  source  sequences.  The  index  associated  with  the  parameter 
vector,  or  codeword,  that  most  reduces  the  distortion  between  the  source  sequence  and  a 
reproduction  of  that  sequence  is  transmitted  as  side  information  to  a  decoder.  The 
decoder  matches  that  index  to  the  index  of  a  codebook  of  optimal  coefficient  vectors.  The 
selected  optimal  coefficient  vector  is  then  used  to  supply  the  parameters  to  reproduce  the 
original  source  at  the  decoder.  The  closeness  of  reproduced  data  to  original  source 
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sequence  is  made  in  a  mean  squared  error  sense.  The  advantage  of  this  procedure  is  the 
elimination  of  quantizing  of  the  information  sent  that's  not  original  data,  the  side 
information.  Extensive  searching  can  be  required,  when  the  codebook  is  large,  to  find  the 
best  parameter  codewords. 

The  huge  computational  load  incurred  during  the  searching  of  the  codebook  is  the 
main  disadvantage  of  this  approach.  In  addition  to  various  audio  applications,  vector 
quantization  is  commonly  employed  in  the  compression  of  images.  In  this  latter  case,  the 
source  sequences  are  subarrays  of  pixels  and  the  codebooks  are  vectors  of  pixel  values 
which  can  reproduce  the  source  pixels  (Xue  and  Crissey,  1991).  The  process  achieves 
compression  in  that  the  index  of  the  reproduction  parameters  can  be  specified  in  fewer  bits 
than  the  original  pixels  themselves.  The  following  relationship  for  the  number  of  bits 
required  per  pixel,  or  compression  rate  (r),  illustrates  that  principle: 

r=(R/k)  bits/pixel,  (3.1) 


where  i?=log2(»i)  is  the  quantizer  rate  in  bits/vector,  m  is  the  number  of  codebook  vectors, 
and  k  is  the  number  of  pbcels.  The  input  source  vector  can  take  many  forms  and  a  vector 
of  pixels  is  just  one  example. 


Another  example  involves  character  recognition  from  half-tone  or 
fax  data,  where  vectors  are  arrays  of  bits  that  are  positioned  over  the 
character  positions  and  the  table  (or  codebook)  is  the  alphabet  of 
characters  that  are  being  recognized.  Larger  vectors  and  tables  result  in 
higher  fidelity  for  a  given  amount  of  compression  at  the  expense  of 
increased  computational  resources.  In  scaler  quantization,  each  vector 
consists  of  a  single  data  element.  In  fact,  the  initial  process  of  analog  to 
digital  data  conversion  is  an  example  of  scalar  quantization  (Bookstein  and 
Storer,  1992). 
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C.  MULTIPATH  SEARCH  CODING 

Sequential  quantization,  like  block  quantization,  takes  advantage  of  dependency 
between  samples,  and  it  has  been  shown  that  they  can  provide  better  performance  than  the 
scalar  version  of  quantization.  Two  categories  of  sequential  quantization  exist:  Multipath 
search  coding  and  predictive  coding. 

Multipath  search  coding  (MSC)  is  a  relatively  new  form  of  quantization  being 
studied.  The  motivation  for  its  study  stems  from  the  performance  bounds  predicted  by  the 
rate  distortion  function  for  sources  with  memory  developed  by  Shannon.  Generally,  this 
type  of  quantization,  which  is  of  the  block  type,  has  the  potential  for  approaching  the  rate 
distortion  bound  with  the  least  amount  of  design  complexity.  Trellis,  and  tree  coding 
comprise  the  dominant  types  of  multi-path  search  coding,  though  codebook 
implementations  are  increasingly  used  (Lynch,  1985). 

1.  Tree  /Trellis  Coding 

MSC's  use  future  as  well  as  previous  sample  values  in  order  to  select  a  quantized 
version  of  a  given  input  sample.  They  are  often  called  look-ahead  coders,  or  tree  or  trellis 
encoders  due  to  this  property.  Compression  is  obtained  by  virtue  of  the  fact  that  the 
selected  quantized  version  of  the  input  sample  sequence  is  coded  into  a  binary  channel 
sequence  wherein  each  sample  is  represented  by  one  binary  digit.  The  techniques  of  trellis 
and  tree  coding  make  use  of  geometric  structures  and  are  very  similar  to  the  vectors  of 
vector  quantization  and  the  lists  of  list  coding.  Each  typical  sample  is  stored  as  a  sequence 
of  branches  in  a  tree  structure.  When  a  sequence  is  selected,  its  corresponding  tree  path  is 
transmitted  as  a  binary  sequence,  with  each  bit  corresponding  to  a  direction  at  each 
sequential  node  of  the  tree,  much  like  the  procedure  used  in  Hufihnan  coding.  A  similar 
procedure  is  used  in  trellis  coding  since  the  trellis  structure  is  really  a  truncated  tree 
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structure  (Jayant  and  Noll,  1984).  The  following  example,  (Jayant  and  Noll,  1984), 
illustrates  the  process. 


TIMEn 


Figure  3.1  Delta  Modulation  Code  Tree  (Jayant  and  Noll,  1984) 


The  code  tree  of  Figure  3 . 1  has  a  set  of  nodes  for  each  time  index  n.  The  nodes 
are  indicated  by  the  black  dots  and  the  branches  by  the  horizontal  lines.  Each  node  has  2^ 
=  2  branches,  where  i?  =  1  bit/sample.  In  Delta  Modulation,  the  decision  outcome  of  a 
waveform  encoder  is  input  to  the  tree  coder  and  each  previous  value  is  updated  with  a 
positive  or  negative  step  of  fixed  size.  Figure  3.1  shows  the  possible  DM  outputs,  which 
can  be  input  to  the  tree  coder,  at  either  end  of  the  two-headed  arrow  on  the  left  side  of 
the  figure.  A  branch  letter  or  reconstruction  value,  selected  from  the  alphabet  of 
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reconstruction  values  (-1,  +i)^  is  labeled  on  each  horizontal  branch.  The  tree  is  traced  by 
following  the  branches  (dashed  in  this  case),  which  show  the  DM  output  at  each  time 
instant,  to  the  nodes,  where  the  sum  of  all  the  previous  output  values  is  labeled.  The 
dashed-line  branches  are  labeled  with  sample  input  values,  and  the  path  is  located  by  a 
binary  sequence  called  a  path  map  (+1,  +1,  -1  for  this  example).  The  nodes  with  the  bold 
numbers  show  the  result  at  each  time  instant  (Jayant  and  Noll,  1984).  The  bits 
corresponding  to  the  reconstruction  values  are  transmitted  and  determine  the  output 
sequences  generated  at  the  decoder.  These  bits  lay  out  a  path  to  follow  through  the  tree. 
The  decoder  has  the  same  tree  structure  and  can  follow  the  branches  to  each  node,  in  a 
process  that  basically  integrates  the  map  sequence,  to  determine  the  correct  value  which 
was  input  to  the  tree  coder.  Unlike  codebook  coding,  tree  and  trellis  coding  can  be  done 
on  a  sample-by-sample  basis  instead  of  on  a  block  basis.  This  is  very  similar  to  maximum- 
likelihood  or  Viterbi  decoding  of  convolutional  channel  codes.  This  latter  process  is  used 
in  the  newest  standard  for  cellular  phone  communications.'  Code  Division  Multiple  Access 
(CDMA),  which  has  wide  ranging  audio  compression  employment.  A  full  treatment  of 
trellis  and  tree  coding  is  available  in  (Lynch,  1985). 

2.  Codebook  Coding 

Codebook  coding  (also  known  as  list  coding)  involves  the  use  of  a  codebook  of  2” 
highly  probable  «-sample  sequences.  The  codebook  vectors  are  highly  probable  since,  in 
image  coding,  an  image  is  scanned  and  used  to  train  the  program  used  to  develop  the 
codebook  to  produce  reproduction  vectors  which  are  “highly  probable”.  One  sequence  is 
selected  from  the  codebook,  that  minimizes  the  distortion  between  itself  and  the  actual 
input  sequence  (Lynch,  1985)  and  (Cappelini,  1985).  The  index  of  the  selected  sequence 
is  coded  as  an  «-bit  word  and  sent  to  the  receiver,  where  the  same  codebook  is  stored. 


26 


If  each  sample  has  M  levels,  then  there  are  AT  possible  sequences  but  only  2”  "typical" 
sequences,  giving  a  compression  ratio  of; 


R  = 


log^y) 


(3.2) 


For  example  (Jayant  and  Noll,  1984),  given  a  coder  sequence  of  length  iV  and  ratei? 
bits/sample,  the  total  number  of  unique  codewords  is  given  by: 

J  =  2^.  (3.3) 

A  sequence  of  output  samples,  :  i  =  1,2,..., 2^  is  assigned  to  each  codeword  and  the 
co  debook  is  comprised  of  the  set  of  all  possible  output  sequences.  The  output  sample 
values  are  selected  from  an  “alphabet”  of  reproducing  values.  The  codebook  coder  is  also 
called  a  vector  quantizer.  Jayant  and  Noll  (1984)  describe  the  process,  which  is  the  same 
for  vector  quantization,  as  follows; 

A  codebook  coder  accepts  a  block  x  of  AT  input  samples,  searches 
through  the  codebook  withy  =  2^  entries,  finds  the  output  sequence  best 
matching  the  input  block,  and  transmits  the  corresponding  codeword  index 
igp,  to  the  decoder  in  the  form  of  NR  bits.  The  decoder  looks  up  the 
corresponding  codeword  in  its  codebook  and  releases  the  N  samples  of 
opt  as  the  output  sequence. 


Since  a  codeword  is  transmitted  once  for  every  N  input  sample  block,  the  codebook  coder 
is  also  a  block  coder;  further  evidence  of  its  equivalence  as  a  vector  quantizer. 


27 


D.  PREDICTIVE  CODING 


Predictive  coders  include  the  well-known  techniques  of  delta  modulation  and 
differential  pulse  code  modulation  (DPCh^.  Both  of  these  techniques  predict  the  next 
sample  value  and  then  quantize  the  difference  between  the  predicted  value  and  the  actual 
value.  The  prediction  is  based  on  a  weighted  combination  of  previously  predicted  values. 
Delta  modulation  uses  a  1-bit  quantizer,  whereas  DPCM  uses  a  k-bit  quantizer. 

1.  Delta  Modulation 

Delta  modulation  is  a  predictive  coding  technique  in  which  the  difference  between 
a  given  sample  and  its  predicted  value  is  quantized  into  one  of  two  levels  (-6,  +6).  If  the 
difference  is  positive,  +5  is  coded,  and  if  the  difference  is  negative,  -6  is  coded.  The 
important  feature  of  delta  modulation  is  that  it  allows  only  two  possible  levels  to  be  coded 
and  transmitted.  Thus,  it  is  known  as  a  "1-bit"  system.  Delta  modulation  can  be  done  in 
two  ways:  conventional  and  adaptive. 

2.  Differential  Pulse  Code  Modulation 

In  Pulse  code  modulation  (PCM),  the  original  analog  signal  is  time-sampled  and 
each  sample  is  quantized  and  transmitted  as  a  digital  signal.  Instead  of  quantizing  each 
sample,  in  DPCM,  the  next  sample  is  predicted  and  the  difference  between  the  actual  and 
predicted  values  is  quantized.  This  is  also  the  basis  of  delta  modulation,  thus  many 
similarities  exist  between  DM  and  DPCM.  In  DPCM,  the  predicted  value,  which  is 
obtained  from  previous  predicted  values  and  differences,  is  also  available  at  the  receiver, 
since  the  identical  predictor  is  used  there.  In  many  applications,  a  more  accurate 
prediction  can  be  obtained  if  more  than  just  the  previous  sample  is  used.  This  is  to  be 
expected  since  many  data  sources  produce  sequential  samples  that  are  not  independent. 
The  number  of  previous  samples  to  use  for  prediction  and  the  predictor  function  itself 
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depend  upon  the  statistical  properties  of  the  data  source.  Adaptive  DPCM  (ADPCM)  is 
frequently  employed  to  allow  non-stationaiy  signals  to  be  tracked  by  the  compression 
algorithm.  In  the  next  chapter,  a  more  detailed  examination  of  the  ADPCM  scheme  is 
conducted. 
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IV.  LOSSLESS  ADAPTIVE  DIFFERENTIAL  PULSE  CODE  MODULATION 


A.  INTRODUCTION 

Normally,  when  designing  a  DPCM  system,  it  is  assumed  that  the  input  data  is 
stationary.  Thus,  a  predictor  and  quantizer  are  designed  with  fixed  parameters.  But  when 
the  input  data  is  non-stationary,  these  fixed-parameter  designs  show  inconsistent  and 
generally  poor  performance  with  respect  to  signal-to-quantizing-noise  ratio.  Adaptive 
designs  have  been  used  effectively  in  these  cases  and  the  approach  boils  down  to  one  of 
three  choices:  an  adaptive  predictor  with  a  fixed  quantizer,  a  fixed  predictor  with  an 
adaptive  quantizer  or  an  adaptive  predictor  and  quantizer.  In  the  ADPCM  comparison  to 
follow,  an  adaptive  predictor  with  a  fixed  quantizer  will  be  simulated. 

Acoustical  signal  digitization  uses  64  kb/s  PCM  (8  bits  per  8  kHz  sample),  in 
communication  networks.  For  efficiency,  the  transmission  rate  is  reduced  to  32  kb/s  (4 
bits  per  8  kHz  sample)  with  ADPCM  coding.  Redundancy  removal  is  accomplished  by 
subtracting  a  predicted  value  from  each  input  sample,  and  entropy  reduction  is  achieved  by 
quantizing  the  difference  between  the  input  data  sample  and  the  predicted  value  to  a 
limited  number  of  amplitude  levels.  Speech  contains  a  relatively  high  short-term 
correlation,  therefore  the  power  of  the  prediction  error  is  less  than  that  of  the  original 
signal  and  fewer  bits  are  required  to  transmit  the  error  signal.  This  is  what  makes 
ADPCM  especially  useful  for  audio  data  compression  applications.  Adaptive  predictors 
are  now  a  major  component  of  32  kb/s  DPCM  systems,  and  instrumental  in  obtaining  the 
best  performance  in  differential  encoding  systems  below  32  kb/s. 

The  principal  components  of  the  ADPCM  system  are  the  quantizer,  the  binary 
encoder/decoder  pair,  and  the  predictor,  shown  in  Figure  4.1.  For  the  data  compression 
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system  simulation  herein,  the  quantizer  is  a  symmetric,  21-level  one  with  fixed  step  size. 
The  quantizer  is  implemented  in  Matlab  code  using  the  round  function,  which  takes  the 
input  values  and  rounds  them  towards  the  nearest  integer.  The  difiference  between  the 
original  prediction  error  and  the  quantized  prediction  error  from  the  quantizer  output  is 
known  as  the  quantization  noise  and  represents  the  lossy  part  of  the  system  which  can  not 
be  recovered.  However,  in  the  adaptive  systems  implemented  in  this  thesis  input  values 
were  also  rounded  so  that  only  integer  data  values  were  used.  This  form  of  lossless 
compression,  using  integer  values,  was  used  since  data  stored  on  a  computer  can  only  be 
represented  to  a  limited  finite  precision.  In  addition,  receiver  outputs  were  sent  through  a 
round  function.  As  a  result,  these  ADPCM  systems  are  lossless  and  quantization  noise  is 
eliminated. 

In  this  chapter  the  ADPCM  predictor  systems  used  for  the  performance 
comparison  are  examined  and  the  various  system  components  presented.  The  finite 
impulse  response  and  infinite  impulse  response  predictors  are  developed  and  the  process 
of  their  underlying  algorithm,  the  least  mean  square  algorithm  is  explained.  Next,  the 
modular  function,  added  to  the  basic  ADPCM  system  to  reduce  the  size  of  the  codebook, 
is  introduced. 

B.  FINITE  IMPULSE  RESPONSE  FILTER 

Figure  4. 1  depicts  the  block  diagram  of  an  ADPCM  system.  These  systems  fall 
into  one  of  two  types,  those  that  have  a  finite-duration  impulse  response  (FIR)  and 
those  that  have  an  infinite-duration  impulse  response  (HR),  (Proakis  and  Manolakis, 

1992).  These  two  types  of  systems  will  be  the  basis  of  this  thesis'  comparative  analysis. 
The  binary  coder  block  in  Figure  4.1  performs  lossless  source  coding  of  the  residual 


32 


sequence,  eq{k).  The  coder  assigns  a  binary  word  of  length  log2  (2L)  to  each  quantization 
level,  X,  on  a  sample-by-sample  basis.  The  Huffinan  coding  algorithm  will  be  utilized  to 
implement  the  binary  coder.  The  box  in  Figure  4. 1  labeled  Piz)  represents  the  adaptive 
predictor.  The  predictor  structure  is  chosen  to  emulate  an  assumed  model  of  the  input 
signal  process.  In  this  thesis,  P{z)  is  chosen  to  have  a  FIR  structure  defined  as: 

P(z)  =  (4.1) 

ui 


where  a,  represents  the  predictor  weights,  based  on  the  assumption  that  speech  can  be 
modeled  appropriately  by  a  linear  prediction  model.  The  transmitter  portion  of  this 
structure  is  an  all-zero  model.  It  utilizes  a  finite  impulse  response  filter  (FIR),  or  a 
moving  average  (MA)  filter,  to  predict  output  values. 

In  Figure  4.1,  the  transfer  function /f(z),  fi'om  eq(k)  to  s(k/k-l),  is  derived  as 
shown  below. 


Z[s(k/k-l)J  =  P(z)EQ(z). 


Which  leads  to: 


H(z)  - 


Z[s(k/(k-l))] 

Em~ 


(4.2) 
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Figure  4.1  FIR  ADPCM  system 

Piz)  represents  the  transfer  function  of  the  MA  predictor  and  H{z)  is  the  transfer  function 
of  the  transmitter  portion  between  eq(k)  and  s(klk-\)  which  has  an  FIR  structure. 

The  adaptive  predictors  of  both  the  transmitter  and  receiver  of  Figure  4. 1  use  the 
least  mean  square  (LMS)  algorithm  which  determines  the  predictor  weights  by  minimizing 
the  instantaneous  square  of  the  error.  This  algorithm  is  used  to  allow  the  gradient  of  the 
error  vector  to  be  estimated  from  available  data  since  no  prior  knowledge  of  the  input 
signal  correlation  matrix,  R,  and  the  cross-correlation  vector  between  the  input  and  the 
desired  response,  is  available.  As  a  result,  the  weight  coefficients  of  the  predictors  are  not 
initially  optimal  weights  like  those  derived  using  Wiener-Hopf  equations,  though  the 
predictor  weights  do  eventually  converge  to  the  optimal  weights  in  stationary 
environments.  However,  the  LMS  algorithm  performs  far  fewer  calculations  as  a  result  as 
it  does  not  require  computation  of  matrix  inverses.  For  the  FIR  ADPCM  implementation, 
the  standard  LMS  algorithm  will  be  used.  Haykin  (1991)  provides  a  detailed  explanation 
of  this  process  which  results  in  three  basic  relations: 
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1.  Filter  Output: 


v(k)=cJL(k)eq.(k). 


(4.3) 


where  eq(k)=[eq(k-\).  eq(k-2), eq{k-M)Y  a  vector  of  length  A/,  v{k)  is  the  output  of 
the  predictor,  and  c(JC)  is  the  vector  of  filter  coefficients  which  are  updated  after  each  input 
sample. 

2.  Estimation  error; 

e(k)’‘S(k)-v(k),  (4.4) 

3.  Predictor  weight  update: 

£(k^l)=£(k)*fiS3.(k)eq(k).  (4.5) 


The  step  size  fi  is  used  to  control  the  updates  to  future  weight  coefficients.  The  bounds 
on  this  variable  are  derived  from  the  bounds  for  updating  the  steepest-descent  algorithm. 
In  that  case,  the  steepest  descent  algorithm  converges  when: 


0  <  n  < 


1 

MR^(0y 


(4.6) 


where  A/ is  the  length  of  the  predictor  and  (0)  is  the  autocorrelation  of  the  input 

sequence.  However,  for  the  LMS  algorithm  an  initial  value  for  fi  is  usually  chosen  as  10% 
of  the  maximum  value,  obtained  in  Equation  4.6. 


35 


C.  INFINITE-IMPULSE  RESPONSE  FILTER 

In  contrast  to  the  standard,  FIR  ADPCM  implementation,  the  DR  filter  system 
output  is  weighted  by  the  linear  combination  of  the  past  input  and  output  samples  instead 
of  just  a  finite  number  of  past  input  samples.  This  fact  gives  rise  to  the  notion  of  infinite 
memory  or  infinite  impulse  response.  To  accurately  model  the  vocal  tract,  the  model 
should  contain  zeros  as  well  as  poles,  and  an  ADPCM  system  transmitter  based  on  a  pole- 
zero  model  is  shown  in  Figure  4.2.  Its  transfer  function,  H{z),  from  eq(k)  to 
v(k)=sQc/k-\)  is  derived  from  the  following  equations: 

v(z)  =  B{z)EQiz) 

*  [Biz)EQ(z)  .  EQiz)Ui2)  *  A{z)[v{z)  -  B{z)EQiz)\, 

Which  leads  to; 

v(z)[(l  -  A(z)J  =  [A(z)  .  B(z)JEQ(z). 

Therefore, 

■  EQ(z)  '  1  -  A(z)  ‘ 


(4.7) 


ENCODER 


DECODER 


Figure  4.2  IIR  ADPCM  System 

Thus,  it  is  a  standard  IIR  predictor  where  zeros  and  poles  can  be  individually  specified. 

For  the  IIR  filter  implementation,  a  version  of  the  International  Telephone  and 
Telegraph  Consultative  Committee  (CCITT)  G.721  recommendation  is  used  (CCITT, 
1984).  The  basic  LMS  algorithm  has  been  modified  to  provide  additional  system  stability 
restraints,  ensure  S3mchronization  between  the  transmitter  and  receiver,  and  prevent  drift 
of  the  AR  portion  of  the  receiver  (Bonnet  et  al,  1990).  The  block  labeled  ‘A’  in  Figure 
4.2  represents  the  AR  portion  of  the  system  and  the  ‘B’  block  represents  the  MA  portion. 
The  modification  to  the  basic  LMS  algorithm  made  to  the  G.721  algorithm  lies  in  the 
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addition  of  leakage  factors,  and  62,  the  stability  constraints  on  the  AR  predictors  weight 
coefficients,  and  the  definition  of  the  signal  p(k). 

The  effect  of  the  modifications  are  to  allow  the  updating  of  the  AR  portion  in  the 
same  fashion  as  the  MA  portion  is  updated,  i.e.,  like  a  transversal  filter  vice  a  recursive 
one  (Bonnet,  1990).  The  stability  constraints  are  explained  below.  Last,  the  signal p(k) 
improves  adjustment  of  the  receiver  onto  the  transmitter  and  allows  resynchronization  of 
the  receiver  to  the  transmitter  in  the  presence  of  transmission  errors.  Bonnet  (1990) 
defines  the  signal  p(k)  in  the  z-domain  as; 

P(z).[l-A(zmz).  (4.8) 

where  ^1^  represents  the  input  to  the  AR  predictor.  As  a  result  of  these  modifications  to 
the  basic  LMS  algorithm,  the  modified  CCITT  G.721  update  equations  become: 

aj(n^l)=(l-bj)aj(n)^a  iSgn(pJsgn(p^j) 

a2(n*lHl-b^a2(n)^<i^gn^J[sgn(p^^-f(aj(n))sgn(p^^^  (4.9) 

b/n*lMl-bj}b/nM ^gn(eqjsgn(eq^) 

where  aj(n^\)  and  a/n+1)  represent  the  recursive  weights  for  a  length  two  AR  predictor, 
*/«+!)  represents  the  jth  weight  coefficient  for  a  length  sk  MA  predictor,  and  sgn(*) 
stands  for  the  sign  function.  Note  that  the  parameters  aj  and  perform  the  same 
function  as  p  does  in  Equation  4.5.  Similar  equations  hold  at  the  receiver.  The  function 
f(n)  is  used  as  a  stability  constraint  to  control  the  boundedness  of  the  AR  parameter  and 
its  range  of  values  are  given  by; 
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m  - ^ 

l  2sgn(a)  otherwise. 


(4.10) 


Note  that,  the  value  of  0.5,  Equation  4.10,  was  selected  by  Bonnet  et  al.  to  correspond  to 
a  limit  on  Oj  for  frequencies  in  the  middle  of  the  telephone  bandwidth  (Bonnet,  1990).  This 
algorithm  was  modified  from  a  standard  LMS  algorithm  by  the  addition  of  the  leakage 
factors  6j  and  These  terms  are  used  to  control  the  drift  in  the  AR  parameters  at  the 
decoder.  Nominal  values  for  6j  and  62^  and  the  updating  factors,  and 

=  2  X  l(r\  62  =  2  X  10-^ 
a 2  =  3.2  X  l(r\  a2  =  2  X  10-\ 

Actual  values  used  for  the  HR  varied  by  a  factor  of  100  or  1000  depending  on  the  signal 
and  are  given  in  the  following  chapter  when  the  signals  are  introduced.  Additionally,  the 
actual  values  of  a;  and  are  limited  to  the  following  range  to  ensure  stability. 

kjl  0.75,  |a,|  ^  (4.12) 

D.  MODULAR  ARITHMETIC  FUNCTION 

The  standard  ADPCM  compression  system  transmits  the  quantized  error  signal. 
The  amount  of  data  compression  achievable  is  dependent  upon  the  entropy  of  the  error 
vector.  The  first-order  entropy,  corresponding  to  a  coder  which  encodes  each  error, 
eq(n),  separately,  is  given  by: 


where  P*,(j)  is  the  relative  frequency  of  the  symbol  eq(k)=j  and  Q  is  the  number  of  discrete 
integer  values  assumed  by  the  input  eq(k)  (Einarsson,  1991).  The  entropy  of  the  error 
sequence  can  usually  be  reduced  using  a  modular  function  such  as  the  one  depicted  in 
Figure  4.3.  This  figure  shows  the  output  of  the  ADPCM  predictor,  eq(k),  as  the  input  to  a 
residual  fimction  which  produces  the  integer-valued  sequence,  v(^).  This  sequence  is  then 
input  to  the  modular  function,  which  uses  modular  arithmetic  to  generate  integer- valued 
data,  The  lossless  procedure  presented  by  Einarsson  to  reduce  the  size  of  the 
codebook  is  a  two-step  procedure. 


^eq(k) , 


ADPCM 
XMITTERI 


v(*) 


mod  Q 


Huffman 

Coder 


SCk) 


kDPCM 

RCVR 


mod  Q\ 


R.2iz)  ^ 


Huffmaq 

Decoder 


iq(k) 


Figure  4.3  ADPCM  system  with  Modular  function 

The  modular  coding  process  described  by  Einarsson  (1991)  starts  with  a  positive 
data  sequence,  eq(k)  k- ...  -1,0,1,2....,  constrained  in  a  range  of  Q  discrete  values.  The 
next  step  of  the  procedure  is  to  generate  a  sequence  of  integer-valued  residuals  v(A:),  k  = 
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...  -1,0,1, 2....,  constrained  in  a  range  of  Q  discrete  values.  The  next  step  of  the 
procedure  is  to  generate  a  sequence  of  integer-valued  residuals  v(lc),  k= ...  -1,  0,  1, 2, ..., 
referred  to  as  the  shifted  sequence.  This  step  is  accomplished  through  the  application  of  a 
linear  filtering  operation  between  the  input  data,  eqQ^  and  a  linear  filter,  R,  with 
coefficients  fo=\  and /i=  -1.  The  filter  i?(z)  forms  the  first  difference  of  eq(J^  and  is  given 
by: 

v(k)  =  eq(k)-eq(k-l).  (4.14) 

Then  the  modular  arithmetic  function,  mod  Q,  is  applied  to  the  sequence  v(^)  to  reduce  it 
to  the  appropriate  range  of  QtoQ-\  before  it  is  entropy  coded.  Table  4. 1  depicts  the 
process.  Therein,  sample  data  values  for  the  input,  eqik),  are  given,  shifted  to  make  them 
all  positive,  filtered  through  the  residual  filter  R,  and  processed  by  the  modular  function  to 
generate  the  resulting  “modular  sequence”  The  binary  coding  process  is  external  to 
this  procedure  and  is  not  shown.  At  the  receiver,  the  linear  filter  operation  is  reversed  and 
the  modular  operation  is  applied  exactly  as  before  resulting  in  the  same  original  data 
values. 

Note  however  that  the  shift,  (+4)  and  (-4),  illustrated  in  Table  4.1,  is  not  required 
and  this  step  could  have  been  skipped.  This  shift  was  done  to  match  the  procedure  given 
by  Einarsson  exactly,  by  starting  with  all  positive  data.  The  algorithm  to  reconstruct  the 
original  data  at  the  receiver  is  derived  from: 

iq(k)=[w(k)-(y(k)-y(k-l))JmodQ.  (4. 15) 

where  eq(A^)  is  the  reconstructed  eq{k).Th.Q  entropy,  H,  of  w(A:)  is  given  by: 

=  -^Pif0)togP^0X  (4.16) 

J-o 
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1 

-5 

7 

1 

-3 

-4 

0 

-1 

11 

0 

-4 

6 

10 

10 

10 

10 

6 

TABLE  4.1  Modular  Coding  Process 


where  (/)  is  the  frequency  of  occurrence  of  the  symbol  Einarsson  showed  that 

the  entropy  for  the  modular  function  residual,  is  lower  than  that  for  v(^).  For 
0=256,  the  residual  \{ky=eq(k)-eq{]c-l)  will  take  on  integer  values  in  the  range  (-255, 
255).  Thus,  eq{K)  =  239,  and  eq(k)  =  -17  reduced  modulo  256  both  result  in  v(^)  =  239. 
Therefore  the  entropy  of  vQc)  is  the  sum  of  two  terms  as  shown  in  Equation  4. 18, 

=  Pv(ii>^ogP^0])  -  Py02>logP^(j2>’  (4.17) 


where  Vj(A:)=yi  and  V2(Jc)=j2  both  result  in  ^*ik)=J.  Writing  Equation  4.17  in  terms  of  the 
entropy  for  i?(A)  gives  the  single  term  Equation: 
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log  PJ). 


(4.18) 


where  PJj)=PJji)  +  PwQt)-  Thus,  since  for  a,b>0 

(a  +  b)log(a  *  b)  >  a  log(a)  +  b  log(b),  (4.19) 

then  H^<  Despite  this  theoretical  advantage  of  reduced  entropy,  Einarsson  states  that 

the  difference  is  usually  negligible.  The  real  advantage  of  using  the  modular  coding 
function  is  a  decrease  in  the  size  of  the  Huffinan  codebook  necessary  to  encode  the 
transmitted  error  vector,  which,  in  turn,  implies  that  fewer  bits  will  be  required.  This 
modular  function  is  incorporated  into  both  the  FIR  and  IIR  versions  of  the  ADPCM  data 
compression  systems  whose  performances  are  compared  in  Chapter  5. 
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V.  PERFORMANCE  COMPARISON 


A.  OVERVIEW 

Generally,  performance  comparisons  of  ADPCM  compression  systems  have  been 
made  using  PCM  systems  as  a  baseline;  often  using  the  signal  to  quantizer  noise  ratio  as  a 
basic  parameter  of  comparison.  When  directly  comparing  ADPCM  coders,  the  mean 
square  error  (MSE)  is  usually  used  since  these  coders  are  designed  to  minimize  this 
parameter.  This  measurement  would  be  valid  for  the  FIR  coder  as  it  is  designed  to 
minimize  the  MSE.  However,  due  to  the  modifications  of  the  LMS  algorithm 
incorporated  into  the  G.721  HR  coder  reviewed  herein,  the  HR  coder  doesn't  minimize 
the  MSE  but  the  quantity  given  by  the  following  relation: 

E(eq/  .  i.  E(^  AJ.  (5.1) 

cc 

where  E(»)  is  the  expectation  operator,  <ar  is  a  positive  adaption  parameter,  <5  is  a  positive 
leakage  factor  and  A„  is  the  vector  of  the  AR  parameters  at  time  n  (Bonnett,  1990).  The 
first  term  of  Equation  5.1  represents  the  MSE  of  the  system  while  the  second  term  is  a 
weighted  version  of  the  AR  predictor  parameters.  Thus,  comparing  MSE's  between  the 
FIR  and  IIR  coders  would  not  be  completely  accurate.  Of  course,  MSE's  are  just  one 
parameter  that  is  frequently  compared  in  ADPCM  comparisons.  Others  include 
compression  ratios,  sound  fidelity,  variances,  compression/decompression  time  and  SNR's. 
In  fact,  no  single  measurement  is  sufficient  to  completely,  reliably  and  easily  classify  the 
coders  performance.  This  is  one  reason  for  the  importance  of  a  variety  of  studies 
investigating  all  the  relevant  properties  of  ADPCM  coders.  In  this  report,  the  variance  of 
the  original  signal  is  compared  to  the  variance  of  the  quantized  error  signal.  Additionally, 
the  compression  ratios  of  each  type  of  coder  is  also  compared.  The  compression  ratio  has 
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many  definitions;  it  is  defined  in  this  thesis  as  the  number  of  bits  needed  to  encode  the 
original  signal  in  a  PCM  format  divided  by  the  number  of  bits  used  to  encode  the 
transmitted  error  sequence.  The  Huffinan  coder  is  utilized  to  encode  the  quantized  error 
and  to  determine  the  compression  ratio  achieved. 

B.  PRESENTATION  OF  DATA 

Five  data  signals  were  examined.  Four  of  them  were  audio  signals  and  the  fifth  is  a 
sinusoid,  which  was  included  since  it  is  nonpersistently-exciting,  and  is  of  the  type  most 
difficult  for  the  IIR  ADPCM  compressor  to  handle  (Bonnet,  1990).  The  original  signals, 
shown  as  the  middle  image  in  Figures  5. 1-5. 5,  are: 

1. )  PFREE  -  a  fast-paced  pop  song  with  a  male  voice  and  music, 

2. )  GUITAR  -  a  musical  selection  played  on  a  guitar, 

3. )  VOICE  -  a  male  voice  speaking  in  a  normal  tone 

4. )  TRANSIENT  -  a  filtered  version  of  a  male  voice  speaking, 

5. )  SINE  -  a  sinusoid  generated  from5(A:)  =  sin(27i  4/100  ^),  k=l, ...,  7025, 
where  k  is  the  number  of  points  in  the  signal.  All  the  signals  had  a  duration  of  7025 
points.  Each  signal  had  a  zero  mean.  The  upper  plot  in  each  Figure  is  the  IIR  receiver 
output  signal  and  the  lower  plot  represents  the  FIR  receiver  estimate  of  the  original  signal 
shown  in  the  middle  plot.  In  every  case  the  received  signal  is  an  exact  integer 
representation  of  the  input  data  after  about  three  filter  lengths.  Note  that  the  HR  predictor 
was  implemented  with  different  values  than  the  nominal  values  proposed  by  Bonnet  for  the 
leakage  factors,  6^  and  dj,  and  update  control  variables,  and  <3:2,  given  in  Chapter  4, 
Equation  (4.11)  and  repeated  here  as  Equation  5.2: 

=  2  X  10-^  ^2  =  2  X  10-^ 

dj  =  3.2  X  10-\  a2  =  2  X  10-\ 
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When  using  these  nominal  values  for  leakage  and  update  factors,  the  compression 
achieved  in  the  ADPCM  coder  was  significantly  lower  for  the  HR  version  than  for  the  FIR 
version.  As  a  result,  these  values  were  experimentally  determined  to  produce  the 
maximum  compression  in  the  quantized  error  power.  Each  actual  value  used  was 
increased  by  a  factor  of  100  for  the  PFREE,  GUITAR,  and  VOICE  signals,  a  factor  of 
1000  for  the  TRANSIENT  signal  and  were  unchanged  for  the  SINE  signal.  The  FIR 
predictor  used  an  update  control  variable,  p,  experimentally  picked  to  provide  the  best 
compression  for  each  signal  and  a  filter  order  of  six  in  each  case.  The  actual  values  were 
4  X  10^,  6  X  10"^,  10'*,  5  X  10'*,  and  6  x  10’*,  listed  in  the  same  order  as  the  corresponding 
signal.  The  filter  order  of  six  was  used  for  two  reasons.  First,  to  make  it  equal  to  the 
order  of  the  AR  portion  of  the  HR  coder  which  also  uses  a  predictor  of  order  sbc. 
Naturally,  this  was  not  done  in  expectation  of  producing  similar  results  but  rather  as  a 
convenient  starting  point  to  experiment  with.  Secondly,  experiments  with  predictors  of 
order  2,  4,  5,  7,  8,  and  10  all  provided  inferior  results  in  terms  of  reducing  the  power  of 
the  error  sequence.  These  experimental  results  were  expected  for  the  filter  orders  less 
than  six  but  were  inexplicable  for  filter  orders  greater  than  six.  Nonetheless,  the  results 
were  repeatable  and  six  was  determined  to  be  the  best  filter  order  for  this  implementation. 
Time  for  compression  and  decompression  was  significantly  lower  for  the  FIR  model  as 
well. 

The  power  of  the  quantized  error  given  for  each  coder  was  one  of  the  parameters 
measured  to  determine  the  amount  of  compression  achieved  by  each  ADPCM  coder.  All 
five  signals  were  compressed  at  two  different  SNR  decibel  levels.  The  SNR  was  increased 
by  increasing  the  power  of  the  input  signal  while  maintaining  the  noise  level  at  a  constant. 
This  was  done  by  multiplying  the  original  signal  by  a  randomly  selected  scaling  factor  of  2 
to  produce  a  six  dB  increase.  Each  signal  power  was  determined  and  Table  5.1  lists  the 
results.  In  this  table,  the  original  input  signal  power,  j/p ,  and  the  quantized  error 
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power,  shown  as  <7^£.g ,  are  presented.  The  reduction  in  power  is  listed  as  P/R.  Greater 
compression  is  represented  by  a  higher  percentage  of  power  reduction,  in  the  ADPCM 
coder  itself.  The  power  reduction  (in  percentage)  is  determined  by: 

(5,3) 

^I/P 

The  fifth  and  eleventh  columns,  respectively,  show  the  power  reduction  for  the  HR  and 
FIR  coders.  A  direct  comparison  between  these  columns  shows  that  the  FIR  coder 
performed  better,  higher  PIR,  in  this  category  than  the  HR  coder,  though  both  coders 
significantly  reduced  the  signal  power  as  compared  to  the  original  signal  variances.  Of 
particular  note  is  the  large  difference  in  compression  between  the  two  systems  for  a 
sinusoidal  input.  Bormet  (1990)  prefaced  this  result  by  noting  that  the  DR  is  less  effective 
on  nonpersistently  excited  signals.  In  addition  to  measuring  the  power  of  each  signal,  the 
transmitted  quantized  error  signal  was  input  to  the  modular  coding  function  and  its  output, 
w,  coded  using  Huffman  coding  to  determine  the  compression  ratios  (CR)  achieved  on 
each  signal.  Again,  a  higher  compression  ratio  implies  better  compression  than  a  lower 
ratio,  where  compression  ratio,  CR,  is  defined  by: 

^  total  no.  of  sample  points*8 
total  no.  of  bits  used 


For  the  first  four  signals  the  compression  ratio  achieved  in  the  HR  is  superior,  as  shown  in 
Table  5.1-  column  three  versus  column  nine,  with  the  only  exception  being  the  sinusoid  at 
both  dB  levels.  The  reduced  signal  power  of  the  error  sequence  output  fi'om  the  FIR 
coder,  allowing  fewer  code  words  to  be  generated  by  the  Huffinan  coder,  would  tend  to 
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explain  this  exception.  In  the  middle  section  of  Table  5.1,  colunms  six  and  seven,  the 
compression  ratios  achieved  by  sending  the  quantized  error  signal  through  the  Huffinan 
coder  without  using  the  modular  function  are  listed.  This  section  is  included  to  allow  a 
tabular  comparison  of  the  compression  achieved  Avith  and  without  the  modular  function. 
Thus  column  three  compares  directly  to  column  six,  for  the  FIR,  and  column  seven  to 
column  nine,  for  the  HR  coder.  In  every  instance  of  comparison  between  the  HR  systems 
and  all  but  two  cases  of  comparison  between  the  FIR  systems  (Transient  and  Sine  signals 
at  lower  db  level)  it  can  be  seen  that  the  CR  achieved  going  through  the  modular  function 
is  greater.  This  result  agrees  with  Einarsson’s  (1991)  paper  wherein  he  stated  that  the  CR 
might  theoretically  increase.  However,  in  most  of  the  cases  displayed  in  Table  5. 1,  the 
modular  function  CR  was  significantly  better,  exceeding  Einarsson’s  projections  of  only 
modest  CR  increases.  As  stated  above,  the  compression  ratio,  CR,  was  determined  by 
totaling  the  number  of  bits  used  to  encode  the  residuals  and  dividing  that  value  into  the 
number  of  bits  required  to  encode  an  8-bit  PCM  signal. 

Table  5.2  (HR  coder)  and  5.3  (FIR  coder)  show  the  results  of  the  comparison  of 
a  number  of  other  parameters,  including  average  wordlength,  L^„  entropy,  (H),  relative 
redundancy  ratio,  (RK),  and  maximum  compression,  (R„a^.  All  of  the  values  used  in  both 
Table  5.2  and  Table  5.3  come  from  the  higher  SNR  data  set  of  Table  5.1.  Specifically, 
Ci?’s  obtained  for  the  HR  implementation  at  the  higher  SNR  level  in  Table  5.1,  column 
three,  coincide  with  the  modular  sequence  values  shown  in  Table  5.2,  CR  row.  Likewise, 
compression  ratio  values  obtained  for  the  higher  SNR  levels  in  the  FIR  implementation  of 
Table  5.1,  column  seven,  correspond  to  the  values  in  the  CR  row  of  Table  5.2.  The  tables 
are  provided  solely  to  assess  the  effectiveness  of  Einarsson’s  modular  function  on  the 
operation  of  each  coder  and  not  to  form  a  basis  of  comparison  between  the  two  coders. 
Thus,  each  table  displays  an  intra-coder  only  comparison  between  vik)  and  eq{]^.  The 
reasoning  for  this  limitation  is  provided  below.  In  both  Table  5.2  and  5.3,  the  results  show 


49 


a  smaller  average  wordlength  for  the  modular  sequence,  as  compared  to  the  error 
sequence,  eqik),  with  the  only  exception  being  in  the  case  of  the  HR  coder  and  the  guitar 
signal  where  the  opposite  is  true.  is  derived  from  the  relation  of  Equation  5.4: 


(5.5) 


where  /,  is  the  length  of  the  codeword  P,  is  the  probability  of  the  codeword  and  M 

is  the  total  number  of  codewords.  This  is  result  is  expected  from  Einarsson’s  (1990) 
conclusions.  In  addition  to  being  smaller  for  yXik),  Table  5.3  (FIR  coder)  shows,  that 
the  entropy,  H,  is  higher,  in  each  case  for  the  FIR  error  sequence,  eq(k),  than  for  the  FIR 
modular  sequence,  w(A).  The  entropy,  defined  in  Equation  2.2  and  repeated  here  as 
Equation  5.6  is: 


M 

H  .  (5-6) 

ul 

This  development,  in  turn  leads  to  the  higher  CR  achieved  on  the  FIR  modular  sequence 
as  compared  to  the  FIR  error  sequence.  In  Table  5.2  (HR  coder),  this  same  relation, 
where  a  lower  if  leads  to  a  higher  CR,  also  holds  true,  again  with  the  exception  of  the 
guitar  signal.  The  guitar  signal  error  sequence  has  a  lower  H  than  does  the  residual 
sequence.  Despite  this  contradiction,  the  CR  is  still  higher  for  the  HR  modular  sequence 
than  it  is  for  the  HR  error  sequence.  The  data  is  insufficient  to  explain  this  anomaly, 
however,  since  nine  of  ten  such  comparisons  (across  both  Tables  5.2  and  5.3)  follow  the 
above  relationship,  it  still  appears  valid.  Relative  redundancy  gives  a  theoretical  measure 
of  how  much  a  signal  can  be  compressed  and  is  higher  for  the  modular  sequence  (than  for 
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the  error  sequences)  indicating  a  greater  compressibility.  The  relative  redundancy,  RR,  is 
defined  by: 


RR  =  J  - 


H 


log^ 


(5.7) 


and  is  related  to  the  maximum  compression  ratio  by  the  following  relation: 

1 


R_ 


I  -  RR’ 


(5.8) 


where  H  is  the  entropy  of  the  signal,  Af  represents  the  number  of  amplitude  levels  in  the 
compressed  signal,  and  is  the  maximum  compression  that  can  theoretically  be 
achieved  on  the  given  signal.  Since  there  are  different  numbers  of  amplitude  levels  in  the 
FIR  and  IIR  signals,  the  M  parameter,  the  utility  of  comparisons  between  Tables  5.2  and 
5.3,  is  reduced.  Each  ADPCM  coder  and  associated  modular  function  reduces  its  error 
sequence  to  Q  integer  levels.  However,  the  Q  value  is  signal  and  coder  system  dependent. 
Therefore,  just  as  comparisons  cannot  be  made  between  the  same  parameters  of  different 
signals,  no  direct  comparison  can  be  made  between  the  RRs  or  R^Js  of  each  coder 
because  they  depend  on  M  which  is  Q  dependent.  Conversely,  comparisons  can  be  made 
between  the  CR  of  each  coder  system  due  to  the  way  CR  is  defined  which  is  simply  as  the 
ratio  of  the  total  number  of  bits  used  to  encode  a  given  compressed  signal  to  the  number 
required  to  encode  the  same  signal  in  PCM  format.  Thus,  comparing  CRs  is  just  a  simple 
comparison  of  total  bits  used,  irrespective  of  other  parameters.  In  a  similar  manner, 
although  better  compression  is  achieved  when  RR  increases  and  likewise  for  the  CR,  these 
two  parameters  are  not  related.  The  RR  is  an  imprecise  theoretical  value  and  is  only  a 
guideline  used  to  determine  a  theoretical  maximum  compression  which  is  usually  never 
reached  (Lynch,  1984).  In  any  case,  RR  is  dependent  on  Af  and  H  and  thus  is  a  direct 
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function  of  compression  achieved  in  the  ADPCM  coder  itself  Alternately,  the  CR  is 
based  on  a  comparison  between  the  actual  number  of  bits  used  in  the  Huffman  coder  and 
the  number  which  would  have  been  required  for  PCM  and  is  only  indirectly  related  to 
coder  compression.  Therefore,  RR  and  CR  can  move  independently  of  one  another.  In 
effect  then,  RRs  only  purpose  is  to  calculate  R^  which  was  determined  for  both  the 
modular  sequence,  ^ik),  output  from  the  modular  function  and  the  original  error 
sequence,  eq(k),  output  from  the  ADPCM  compressors.  This  term  is  included  in  Tables 
5.2  and  5.3  only  to  show  that  the  actual  CR  achieved  in  the  Huf&nan  coders  did  not 
achieve  the  theoretical  maximum  value. 
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MOD  FUNCTION 

|w/0  MOD 

MOD  FUNCTION 

S/N 

HR 

HR 

nR 

HR 

FIR 

FIR 

FIR 

FIR 

dB 

Signal 

CR 

^  EQ 

PIR 

% 

CR 

CP 

^i/p 

CR 

^EQ 

PIR 

% 

20 

Pfree 

2.00 

0.60 

14.0 

0.81 

0.62 

0.70 

1.36 

0.49 

30.0 

20 

Guitar 

1.40 

1.20 

54.0 

0.69 

0.61 

2.60 

1.20 

0.60 

77.0 

26 

Voice 

6.60 

5.20 

48.0 

1.74 

1.88 

9.90 

6.20 

4.63 

53.0 

28 

Trans 

3.20 

11.7 

23.0 

2.95 

3.17 

15.1 

3.10 

6.01 

60.0 

25 

Sine 

0.80 

11.8 

01.0 

0.52 

0.92 

12.5 

0.91 

0.89 

93.0 

dB 

■ 

26 

Pfree 

2.10 

9.30 

37.0 

0.85 

0.68 

14.8 

1.54 

3.79 

74.0 

26 

Guitar 

1.60 

50.0 

59.0 

0.72 

0.65 

122 

1.40 

7.52 

94.0 

32 

Voice 

7.01 

419 

42.0 

2.01 

2.03 

723 

6.06 

314 

57.0 

34 

Trans 

3.10 

792 

52.0 

2.66 

2.08 

1662 

2.10 

612 

63.0 

31 

Sine 

1.10 

19.4 

09.0 

0.51 

0.87 

21.2 

2.00 

4.04 

81.0 

TABLE  5.1  FIR  and  HR  Compression  Ratios  (CR)  and  Power  Reductions  (P/R) 
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SIGNAL 

pfree 

guitar 

voice 

transient 

sine 

SYSTEM 

w 

eq 

w 

eq 

w 

eq 

w 

eq 

w 

eq 

D 

^mcDc 

2.78 

4.17 

2.78 

2.86 

14.3 

7.14 

3.13 

3.57 

3.28 

2.13 

RR 

0.64 

0.76 

0.64 

0.65 

0.93 

0.86 

0.68 

0.72 

0.69 

0.53 

1.87 

2.26 

2.41 

2.37 

1.47 

1.91 

1.61 

2.13 

3.20 

3.85 

H 

1.76 

1.78 

2.36 

2.31 

0.19 

0.29 

2.60 

2.74 

2.07 

3.15 

CR 

2.10 

0.85 

1.60 

0.72 

7.01 

2.01 

3.10 

2.66 

1.10 

0.51 

Total  Bits 

(xlO^) 

11.6 

28.4 

30.6 

65.1 

2.20 

19.9 

13.4 

19.6 

22.3 

28.4 

TABLE  5.2  (IIR)  Comparison  of  Coding  Parameters  for  Residual,  Mk),  vs.  eq{k) 


SIGNAL 

pfree 

guitar 

voice 

transient 

sine 

Parameter 

w 

eq 

w 

eq 

w 

eq 

w 

eq 

w 

eq 

3.57 

3.13 

4.55 

2.04 

7.69 

3.57 

3.13 

3.57 

2.27 

3.23 

RR 

0.72 

0.68 

0.78 

0.51 

0.87 

0.72 

0.68 

0.72 

0.56 

0.69 

1.56 

2.17 

1.53 

1.61 

1.05 

1.21 

2.31 

3.17 

1.89 

2.07 

H 

1.46 

2.13 

2.69 

2.96 

0.37 

0.73 

2.14 

2.91 

1.75 

3.01 

CR 

1.54 

0.68 

1.40 

0.65 

6.06 

2.03 

2.70 

2.08 

2.00 

0.87 

Total  Bits 

(xlO^) 

20.8 

23.3 

33.2 

61.6 

5.68 

19.7 

13.4 

17.1 

15.3 

18.4 

TABLE  5.3  (FIR)  Comparison  of  Coding  Parameters  for  Residual,  MJc),  vs.  eq{lc) 
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VI.  CONCLUSIONS 


This  thesis  set  out  to  compare  the  performance  of  two  versions  of  ADPCM  data 
compression  systems.  The  primary  objective  was  to  use  a  set  of  evaluation  criteria  to 
determine,  for  a  given  implementation  and  a  given  set  of  algorithms,  whether  the  FIR 
ADPCM  compressor  performed  better  than  the  UR  ADPCM  compressor  or  vice  versa.  A 
Huffinan  binary  coder/decoder  was  added  to  the  basic  ADPCM  compressors  thus 
completing  an  entire  data  compression  system.  Therefore,  a  comparison  was  also  made  of 
each  associated  binary  coder’s  performance.  The  specific  results  of  the  testing  are  detailed 
in  the  previous  chapter.  It  must  be  noted  that  these  results  are  valid  only  for  the  specific 
implementation  used  in  this  thesis,  which  included  a  FIR  filter  of  order  6  only,  and  do  not 
necessarily  hold  for  different  implementations,  filters  of  different  orders,  or  other  changes. 
As  stated  in  the  previous  chapter,  a  FIR  filter  of  order  six  was  experimentally  picked  on 
the  basis  of  providing  better  compression  in  the  ADPCM  coder  than  a  sampling  of  filters 
using  different  orders. 

Table  5.1  displays  the  results  of  the  main  comparison  between  IIR  and  FIR 
ADPCM  compressors.  The  power  reduction,  (PIR),  percentage  was  used  to  determine 
the  percentage  by  which  each  compressor  reduced  the  original  signal  power.  Clearly,  the 
FIR  performed  far  better  than  the  IIR  in  this  category.  This  fact  was  particularly 
highlighted  by  the  greater  power  reduction  the  FIR  compressor  achieved  using  the 
sinusoidal  signal;  a  non-persistently  excited  signal  known  to  be  problematic  for  the  IIR 
compressor,  (Table  5.1,  column  five  -rows  eight  and  fourteen  vs.  column  eleven-  and  the 
same  rows).  Not  visible  in  any  of  the  tables  was  the  fact  that  the  FIR  coder 
implementation  compressed  each  signal  approximately  40%  faster  than  did  the  IIR  coder. 
As  dramatic  as  this  difference  was,  since  the  IIR  coder  was  not  optimized  for  the  fastest 
possible  speed,  this  result  cannot  be  assumed  to  always  hold  for  other  implementations. 
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The  second  major  analysis  focused  on  how  effectively  the  modular  function  performed  its 
Job.  The  purpose  of  the  modular  function  was  to  reduce  the  size  of  the  codepage  required 
to  transmit  the  compressed  signal.  In  order  to  accomplish  this  task,  the  average 
wordlength,  was  determined;  where  a  smaller  indicated  a  smaller  codepage.  The 
compression  ratio,  (CR),  of  the  binary  coder  was  also  calculated  to  help  facilitate  the 
comparison  of  each  modular  functions’  effectiveness.  Thus,  in  Table  5.2  and  Table  5.3, 
both  of  which  refer  only  to  data  taken  from  the  higher  SNR  set  of  Table  5. 1,  both  the 
error  sequence,  eq,  and  the  modular  sequence,  w,  were  run  through  the  Huffman  coder. 
Table  5.2  shows  the  results  from  the  IIR  coder,  where  the  modular  sequence  had  a  smaller 
than  the  error  sequence  for  all  signals  except  the  GUITAR  signal.  The  FIR  modular 
function  produced  a  smaller  modular  sequence  for  all  five  signals  (Table  5.3). 
Furthermore,  the  CR  of  each  signal  was  greater  for  the  modular  sequence  than  for  the 
error  sequence.  This  increase  in  CR  was  a  theoretical  possibility  foretold  in  Einarsson’s 
work.  This  result  held  equally  well  for  the  FIR  coder  as  well  as  the  IIR  coder.  Thus,  the 
addition  of  the  modular  function  did  improve  the  performance  of  both  ADPCM 
compression  systems  as  postulated  by  Einarsson.  The  compression  ratios  which  are  less 
than  one  show  that  when  the  sequence  was  coded  with  the  Huffman  coder,  the  codepage 
required  to  transmit  the  sequence  actually  expanded  rather  than  become  compressed. 

This  expansion  phenomenon  occurs  when  the  input  sequence  statistics  do  not  match  the 
statistical  expectations  of  the  binary  coder,  resulting  in  a  mismatch  which  causes  the  data 
to  expand  rather  than  compress.  It  is  also  possible  that  this  statistical  mismatch  is  what 
sometime  caused  the  CR  to  move  in  the  opposite  direction  from  what  the  RR  would  cause 
one  to  expect  (Tables  5.2  and  5.3). 

The  limited  data  set  and  restricted  applicability  of  the  data  still  make  the  results 
somewhat  inconclusive  and  further  testing  is  needed  to  completely  specify  each  system 
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capabilities.  This  inconclusivity  is  also  partly  due  to  the  lack  of  a  standard  comparison 
criteria  upon  which  to  definitively  judge  different  types  of  ADPCM  systems.  Additional 
testing  might  concentrate  on  different  types  of  compression  ratios,  see  Lynch  (1985)  for 
more  discussion  on  types  of  compression  ratios,  other  SNR’s  and  certainly  different  types 
of  data  signals  and  filter  orders,  as  starting  points  for  further  research. 
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APPENDIX 


(MATLAB  CODE) 
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A.  FIR  ADPCM  Implementation 

o/o - 

%  FINAL  OUTPUT  IS  SESTl 

%  LMS  of  order  M=6  using  round  for  Q  wrt  error,  mu  values  signal  dependent 
Vo  This  program  simulates  an  ADPCM  compression  scheme  using  an  fir  predictor 
% - - - 


clear 

clg 

k=l;2000; 

%load  input  signal 
mu=X; 

M=6; 

A=20; 

u=A*[zeros([  l,M]),s]; 
ul=round(u); 

N=length(u); 

shat=0; 

w=zeros(M); 

e=0; 

eq=zeros(l,M); 

cw=0; 

minVl=min(ul); 
maxVl=max(ul); 
modl=maxVl-minVl ; 
cl=zeros(l,modl); 

%  implement  LMS  algorithm 
for  n  =  M+l:N, 

eql=eq(n- 1  >1  ;n-M)'; 
shat  =  w(;,n-l)'  *eql; 
e  =  ul(n)  -  shat; 
eq(n)  =round(e); 

0,i]  =  min(abs(Q-eq(n))); 
cl(i)=cl(i)+l; 

w(:,n)=w(:,n-l)+mu*eql*eq(n); 

end 


%no.  of  data  points 

%  step  size  mu 
%  predictor  length 

%  scaling  factor  used  to  change  SNR 
%  i/p  scaled  by  A 
%  round  input 
%  data  length 
%  predictor  output 
%  initial  weights 
%  initial  error 
%  predictor  input 

%  get  minimum  value  of  input  for  H  calc. 
%  get  max  value  of  input  for  H  calc. 

%  determine  mod  function  value 


%  predictor  output 
%  error:  voice  signal 
%  quantizer  output 
%  get  index  of  quantizer  value 
%  update  counter 
%  recalculate  weights 
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W=max(ec[); 

Vl=min(eq); 

mod=W-Vl+l 

minV=Vl; 

V1=-V1: 


%  get  max  value  of  eq  to  det.  mod  value 
%  get  min  value  of  eq  to  det.  mod  value 
%  get  mod  value  for  error  sequence 
%  store  value  of  VI  in  minV  variable 
%  negate  min  value  of  eq  to  make  it  positive 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


o/oyo/oyo/o/o  /o/o/o/oyo/o/o/oyoyoyo/o/o/o/o/o/o/o/o/o/o/o/o/o 


%  This  section  computes  the  probability  vector  for  the  error  vector  eq 
% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

C  l=sum(cl);  %  total  number  of  data  values 

Pe=cl./Cl;  %  vector  of  probabilities 

Pe=sort(Pe);  %  sort  prob  vector  from  low  to  high 

pe=fliplr(Pe);  %  arrange  prob  vector  from  high  to  low 

%  Initialize  Modulus  Function 

c=zeros(l,mod); 

wk=zeros(l,M); 

Ql=[0;l;mod-1]; 

R=zeros(l,mod); 

%  Modulus  Function  which  takes  values  from  -mod  to  +mod  and  reduces  the  range 
%  to  0  to  +mod  to  reduce  the  number  of  codewords  necessary  to  transmit  the 
%  quantized  error  signal 
for  n  =  M+1  :N, 

eqa(n)=V  1  +eq(n); 
wk(n)=eqa(n)-eqa(n- 1 ); 

[j,i]  =  min(abs(Ql-wk(n)));  %  get  index  of  quantizer  value 

R(i)=R(i)+l; 

if  wk(n)  >=  0 

wkl(n)=rem(wk(n),mod);  %  mod  function 
[j,i]  =  min(abs(Ql-wkl(n)));  %  get  index  of  quantizer  value 
c(i)=c(i)+l; 
else 

wkl  (n)=mo  d+rem(wk(n),mod); 

jj,i]  =  min(abs(Ql-wkl(n)));  %  get  index  of  quantizer  value 
c(i)=c(i)+l; 
end 
end 
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%%%%%%%%%%%%%%%%%%%%%%%%%%0/o0/o%%%%0/o%%%0/o0/o%0/^0/^0/g 

%%%%%%%%%%%%0/o0/o%%%%0/o%0/o0/o%0/o%%%%0/o0/o0/o 

% 

%  This  section  computes  the  probability  vector  for  the  mod  function  o/p  w' 

% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,%%%%% 

%%%%%%%%%%%%%%%%%o/o%%%%%o/o%%%%o/o%% 


C=sum(c); 

P=c./C; 

P=sort(P); 

p=fliplr(P); 


%  total  number  of  data  values 
%  vector  of  probabilities 
%  sort  prob  vector  from  low  to  high 
%  sort  prob  vector  from  high  to  low 


%°/0%%%%%%%%%%%%%%%%0/o%0/o%%0/o0/o0/o0/o0/00/„0/^0/^0/^0/^0/„O/^0/g0/„0/j,0/^0/j,0/„0/„ 

%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  This  section  computes  the  probability  vector  for  the  residuals 
% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%0/o0/o%0/o0/o0/^0/^0/^0/^0/^0/^0/^0/^0/^0/g 

%%%%%%%%%%%%%%%%%%%%%0/oO/o 


wc=sum(R);  %  total  number  of  data  values 

Pw=R./wc;  %  vector  of  probabilities 

Pw=sort(Pw);  %  sort  prob  vector  from  low  to  high 

pw=fliplr(Pw); 

%  Implement  Huffman  Encoder/Decoder  using  function  huffinan4.m 
%  which  returns  wkmod-  rcvd  residuals,  CR-compression  ratio, 

%  Eb  -bits  in  error,  and 
%  bits-total  no.  of  bits  used 


Ebl=0; 

CR1=0; 

Bits=0; 

%minV=0;  %  set  to  zero  when  calculating  values  for  w' 


[wkmod,H,L_avg,Bits,E,CR,Lvar,Rmax,eff]=huffinan4(eq,mod,pe,minV); 
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%linplement  FIR  Receiver 
%Imtializations 
L=length(eq); 
wr=zeros(M); 
shatr=0; 

sest=zeros(l,M); 
wkmod  l=zeros(  1  ,M); 

%FIR  Receiver  LMS  algorithm 
forn=M+l:N 

wkmod  1  (n)=Tem(wkmod(n)+wkmod  1  (n- 1  ),mod); 

eqhat(n)=wkmo  d  1  (n)- V 1 ; 

eq2=eqhat(n- 1 :  - 1  :n-M); 

wr(:,n)=wr(:,n-l)+mu*eq2'*eqhat(n); 

shatr=wr(:,n-l)'*eq2';%filter  output 

sest(n)  =  shatr  +  eqhat(n);  %  est  of  noisy  signal  y 


%  data  length 

%  assumed  predictor  coefficients 
%  assumed  predictor  output 
%  assumed  rcvr  output 


end 

sestl=round(sest); 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%% 

% 


%  This  section  computes  signal  to  noise  ratios  and  input  and 
%  output  power  ratios 


%%%%%%%%%%%%%%%%%%%%%%% 

Pn=l 

yo=xcorr(sest, 'biased'); 

yi=xcorr(u, 'biased'); 

ra=length(yi)/2; 

ri=round(ra); 

rb=length(yo)/2; 

ro=round(rb); 

Pi=10*log(yi(ri)) 

Po=10*log(yo(ro)) 

SNRo=Po/Pn 

SNRi=Pi/Pn 


69 


B.  IIR  ADPCM  Implementation 


O/o - 

%  FINAL  OUTPUT  IS  SRHATl 

%  transmitter  uses  backward  adaption  HR.  Rcvr  uses  error  signal. 

%  ARMA  MODEL  MA  part  is  B,  AR  part  is  A. 

%  This  program  simulates  an  ADPCM  compression  scheme  using  an  iir  predictor 

o/o - 

clear 

clg 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%0/o0/o0/o0/o0/o0/^0/^0/^0/^ 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  input  data  section 
% 

%load  guitar,  mat; 

%d=g_synth(l:7025); 

% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


P=length(d); 

P=2; 

%  ar  predictor  length 

q=6; 

%  ma  predictor  length 

A=10; 

%  scaling  factor 

%Initializations 

u=A*[zeros([l,q]),d']; 

N=length(u); 

%  data  length 

k=length(u); 

uhatA=0; 

uhatB=0; 

uhat=zeros([  1  ,q+l  ]); 
e=0; 

%  initial  error 

ul=round(u); 

eql(l:q)=zeros([l,q]); 

%  predictor  input 

u2(l:q)=zeros(l,q); 
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eq=zeros(l,c[); 

P=zeros(l,q); 

shat=zeros(l,q); 

deltal=l-le-6; 

delta2=l-2e-6; 

al=zeros(l,p); 

a2=zeros(l,p); 

b=zeros(l,q); 

alphal=3.2e-6; 

alpha2=2e-6; 

^0; 

minVl=min(ul); 
maxVl=max(ul); 
mo  d  1  =max VI -minVl ; 
Q=(0:modl-1); 
c  1  =zeros(  1 ,  mod  1 + 1 ); 


%  predictor  coefficients 


%  get  minimum  value  of  input  for  H  calc. 
%  get  maximum  value  of  input  for  H  calc. 
%  get  value  to  use  for  modular  function 


%11R  Transmitter  LMS  algorithm 
forn=q+l:N, 

e  =  ul(n)  -  uhat(n); 
eq(n)=  round(e); 

[j,i]  =  min(abs(Q-eq(n))); 
cl(i)=cl(i)+l; 
eql  =eq(n-l:-l:n-q); 
uhatB  =  b(:,n-l)'*  eql'; 

P(n)  =  uhatB+eq(n); 
shat(n)  =  P(n)  +  uhatA; 
shatl=shat(n:-l  :n-l); 
uhatA  =  [al(n-l)  a2(n-l)]*(shatl)';  %predictor  output 
uhat(n+l)  =  uhatA  +  uhatB; 

al(n)=deltal*al(n-l)  +  alphal*sgn(P(n-l))*sgn(P(n-2)); 
if  abs(al(n))  <=  0.5, 
f==4*al(n); 
else 

f=2*sgn(al(n)); 

end 

a2(n)=delta2  *  a2(n- 1  )+alpha2  *  sgn(P(n))  *  [sgn(P(n-2))-f'‘  sgn(P(n- 1))] ; 
if  a2(n)  >  0.75, 

a2(n)=0.75; 


%  error:  voice  signal 
%  get  index  of  quantizer  value 
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elseif  a2(n)  <  -0.75, 
a2(n)=-0.75; 
else 

a2(n)=a2(n); 

end 

if  al(n)  >l-2^(-4)-a2(n), 

al(n)=l-2^(-4)-a2(n); 
elseif  al(n)  <  -(l-2'^(-4)-a2(n)); 

al(n)=-(l-2^(-4)-a2(n)); 

else 

al(n)=al(n); 

end 

b(:,n)=deltal*b(:,n-l)+alpha2*sgn(eql)'*sgn(eq(n-l)); 

end 

clear  al  a2  b  uhatA  uhatB  P 


V=niin(eq); 

niinV=V; 

V1=-1*V; 

W=max(eq); 

mod=W+Vl+l 

wk=zeros(l,q); 


%  get  minimum  value  of  eq  to  shift  with 

%  change  min  value  to  positive 
%  get  max  value  of  eq  to  det.  mod  value 
%  get  mod  value 


%%%%%%%%%%®/o%%%%%%%%%%%%%%%%%%o/o%%%%%o/o%%%%o/o% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  This  section  computes  the  probability  vector  for  the  error  vector  eq 
% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

C 1  =sum(c  1 );  %  total  number  of  data  values 

Pe=c  l./Cl;  %  vector  of  probabilities 

Pe=sort(Pe);  %  sort  prob  vector  from  low  to  high 

pe=fliplr(Pe); 

Ql=[0:l:mod-1]; 

%  Initialize  Modulus  Function 
c=zeros(l,mod); 
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■w=zeros(l,mod); 

%  Modulus  Function  which  reduces  range  toto  0  to  +mod  to  reduce  the  number  of 
%  codewords  necessary  to  transmit  the  quantized  error  signal 

for  n=  q+l;N, 

eqa(n)=Vl+eq(n);  %  adjust  for  min  of  eq 

wk(n)=eqa(n)-eqa(n- 1); 

Ij,i]  =  min(abs(Ql-wk(n)));  %  get  index  of  quantizer  value 

w(i)=w(i)+l; 

if  wk(n)  >=  0 

wkl(n)=rem(wk(n),mod);  %  mod  function 

[j,i]  =  min(abs(Ql-wkl(n)));  %  get  index  of  quantizer  value 
c(i)=c(i)+l; 
else 

wkl  (n)=mod+rem(wk(n),mod); 

[j,i]  =  min(abs(Ql-wkl(n)));  %  get  index  of  quantizer  value 
c(i)=c(i)+l; 
end 
end 

clear  eqa 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  This  section  computes  the  probability  vector  for  the  mod  function  o/p  w' 

% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


C=sum(c);  %  total  number  of  data  values 

P=c./C;  %  vector  of  probabilities 

P=sort(P);  %  sort  prob  vector  from  low  to  high 

p=fliplr(P); 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  This  section  computes  the  probability  vector  for  the  residuals  w 


73 


% 

%%%%%%%%%%%%%%%%%%%%%%%0/o%%%%%0/o%%%%%0/o%0/o%%%% 

%%%%%%%%%%%%%%%%%%%%%0/o%0/o%%0/o0/o%%% 


wc=sum(w); 

Pw=w./wc; 

Pw=sort(Pw); 
pw=fliplr(Pw); 

%lmplemeiit  lER  Receiver 

%Imtiali2ations 
L=length(eq); 
wkmod  1  =2eros(  1 ,  q); 
x(l:q)=[zeros([l,q])]'; 
br=zeros(q); 
shatr(  1 :  q)=zeros([  1  ,q]); 
shatrA=0; 
shatrB=0; 
srhat=zeros([l,q]); 

Pr=zeros(l,q); 
arl=zeros(l,q); 

ar2=2eros(l,q); 

CR1=0; 

Bits=0; 

Ebl=0; 
niinV=0; 

%  Implement  Huffman  Encoder/Decoder  using  function  huffinan4.ni 
%  which  returns  wkmod-  rcvd  residuals,  CR-compression  ratio, 

%  Eb  -bits  in  error,  and 
%  bits-total  no.  of  bits  used 

[wkmod,H,L_avg,Bits,E,CR,Lvar,Rmax,efi]=hufiman4(wkl,mod,p,minV); 
for  n=q+l:N 

wkmod  1  (n)=rem(wkmod(n)+wkmod  1  (n- 1  ),mod); 
eqhat(n)=wkmod  1  (n)-Vl ;  %  adjust  for  min  of  eq 

eq2=eqhat(n- 1 1  :n-q); 


%  total  number  of  data  values 
%  vector  of  probabilities 
%  sort  prob  vector  from  low  to  high 

%  data  length 
%  predictor  input 
%  rcvr  pred  output 

%  rcvr  output 
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%IIR  Receiver  LMS  algorithm 
Pr(n)  =  shatrB  +  eqhat(n); 
srhat(n)  =  Pr(n)  +  shatrA; 
shatrB  =  br(;,n-l)'*(eq2'); 
shatrA=  [arl(n-l)  ar2(n-l)]*(srhat(n-l:-l;n-2))’; 
arl(n)=deltal*arl(n-l)  +  jdphal*sgn(Pr(n-l))*sgn(Pr(n-2)); 
if  abs(arl(n))  <=  0.5, 
f==4*arl(n); 
else 

f=2*sgn(arl(n)); 

end 

ar2(n)=delta2*ar2(n-  l)+alpha2*sgn(Pr(n))*  [sgn(Pr(n-2))-f''sgn(Pr(n- 1))]; 
if  ar2(n)  >  0.75, 

ar2(n)=0.75; 
elseif  ar2(n)  <  -0.75, 
ar2(n)=-0.75; 

else 

ar2(n)=ar2(n); 

end 

if  arl(n)  >l-2^(-4)-ar2(n), 

arl(n)=l-2^(-4)-ar2(n); 
elseif  arl(n)  < -(l-2'^(-4)-ar2(n)); 
arl(n)=  -(l-2^-4)-ar2(n)); 


end 


arl(n)=arl(n); 

end 

br(:,n)=deltal*br(:,n-l)+alpha2’'‘sgn(eq2)'*sgn(eqhat(n)); 


clear  shatrB  shatrA  Pr  arl  ar2  br 
srhatl=round(srhat); 


%  round  output  to  match  rounded  input 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%0/o0/o0/o0/o0/o0/^0/^0/g0/„0/^0/^0/„0/g 

%%%%%%%%%%%%%%%%%%%%%%% 

% 

%  This  section  computes  signal  to  noise  ratios  and  input  and 
%  output  power  ratios 
% 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%%%%%%%%%%%%%%%%%%%%%%% 


Pn=l 

yo=xcorr(srhat, 'biased'); 

yi=xcorr(u,'biased'); 

ra=length(yi)/2; 

ri=round(ra); 

rb=length(yo)/2; 

ro=round(rb); 

Pi=10*log(yi(ri)) 

Po=10*log(yo(ro)) 

SNRo=Po/Pn 

SNRi=Pi/Pn 
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C.  HUFFMAN  Encoding/Decoding  Scheme 

%  HUFFMAN4  finds  the  minimum  variance  Huffinan  code  for  the  symbol 
%  probabilities  entered  by  the  user.  The  algorithm  makes  use  of 
%  permutation  matrices  for  the  combination  and  sorting  of  probabilities. 

%  Permutation  matrices  are  used  because  they  provide  a  convenient  record 
%  of  operations,  so  that  the  codewords  can  then  be  constructed  fairly  easily 
%  once  the  combination  and  sorting  of  probabilities  yields  just  two 
%  probabilities.  At  this  point  a  zero  is  assigned  to  one  of  the 
%  probabilities  and  a  one  assigned  to  the  other.  The  permutation  matrices 
%  are  used  to  append  additional  zeros  and  ones  as  appropriate  to  obtain 
%  the  final  codeword  for  each  symbol.  This  program  both  encodes  and  decodes 
%  Program  revised  to  accept  a  vector  of  data  (wkl)  and  with  (mod)  different 
%  values  and  apply  the  generated  Huffman  code  (generated  as  CW  and  converted 
%  to  decimal-  Cwdr)  to  the  data  and  decode  it  at  the  receiver. 

%  Written  by  K.L.  Frack  for  EC4580  Course  Project%  Revised  by  M.  V.  Cooperwood 
%  for  data  compression  thesis 
%  Last  Update;  20  June  1995 
%  clear  Bits  out  H  eff  RR  Lavg  Rmax 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%INPUT  THE  SYMBOLS  TO  BE  CODED  % 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

%function  [wkmod,H,L_avg,B  its, E,CR,Lvar, Rmax,  eff]=huffman4(wkl  ,mod,p,minV); 
disp('Commence  symbol  probability  determination  process') 

total=length(wkl); 

%  determines  the  number  of  characters  in  the  input  file. 

%  This  section  determines  the  frequency  of  occurrence  of  each  unique  character 
%(As) 

%  in  the  input  file  and  associates  the  frequency  with  the  respective  character 
%(X). 

%nb=input('Input  the  number  of  bins  (128  or  256) '); 

Z=[l:mod]; 

[As,X]=hist(wkl  ,Z); 


%  This  section  determines  calculates  the  integer  length  (1)  of  the  respective  codeword. 
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l=zeros(l,mod);  %  initialize  length  vector 
for 

%  INPUT  THE  NUMBER  OF  SYMBOLS  TO  BE  CODED.  NO  TRIVIAL 
%  SOLUTION  ALLOWED. 

q=0;  %  q  =  number  of  symbols.  Set  to  0  to  ensure  that 

%  the  loop 

%  will  be  executed  at  least  once 

while  q<3  %  Need  at  least  3  symbols  for  a  non-trivial  solution 

q=mod 

%q=input('Enter  the  number  of  symbols:  ');  %allows  for  keyboard  input 
ifq<3, 

beep, 

disp('Trivial  solution.  Use  a  larger  number  ofsymbols.'), 
end 
end 

%  ENTER  THE  SYMBOL  PROBABILITIES.  For  keyboard  input  only 
%  Note:  The  probabilities  must  sum  to  1.00  and  must  be  in  entered  in 
%  descending  order  for  the  algorithm  to  work  properly.  Since  the 
%  algorithm 

%  will  give  erroneous  results  if  these  errors  are  overlooked,  error 
%  checking 

%  routines  are  included  in  later  steps. 

%disp('  ’) 

%disp('Enter  the  symbol  probabilities  ( in  descending  order).') 

%for  =l:q, 

%p(i)=input(['  Enter  the  probability  of  s',int2str(i),':  ']); 

%end 

%  ENSURE  THERE  ARE  ENOUGH  PROBABILITIES  ENTERED 
%  If  <RETURN>  is  inadvertently  struck  before  a  probability  is  entered 
%  the 

%  input  command  could  yield  a  probability  vector  which  is  too  small 
%  This 

%  causes  the  program  to  crash.  This  procedure  prevents  this  from 
%  happening 

%  by  setting  all  of  the  missing  probabilities  to  zero.  In  this  event  the 
%  user  can  correct  the  wrong  probabilities  in  a  later  step. 

%iflength(p)<q, 

%  p=[p;zeros(q-length(p),l)]; 
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%end 

%  ERROR  CHECK  THE  SYMBOL  PROBABILITIES 
correct='n';  %correct  =  'n'  ensures  at  least  once  through  the  error 
%checking 
%  loop. 
count=0; 

%  count  =  0  makes  the  loop  a  little  simpler.  It 
%prevents  %  the 

%  program  from  prompting  for  a  correction  until  the  loop 
%  has  %  been  executed  at  least  once, 

while  correct  ~=  'y'  %  Keep  looping  until  correct, 
if  count>0;  %  This  procedure  will  be  executed  only  if  there  are 
%  errors  to  be  corrected. 

s=input('Enter  the  index  of  the  incorrect  probability:  '); 
p(s)=input(['Enter  the  correct  probability  for  s',int2str(s),':  ']); 
end 

count=l; 
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